Re: snapshot destruction making IO extremely slow
Am 2017-05-24 um 10:23 schrieb Marat Khalili: > Hello, > >> It occurs when enabling quotas on a volume. When there are a >> lot of snapshots that are deleted, the system becomes extremely >> unresponsive (IO often waiting for 30s on a SSD). When I don't have >> quotas, removing snapshots is fast. > Same problem here. It is now common knowledge in the list that qgroups > cause performance problems. I try to avoid deleting many snapshots at > once because of this. I enabled quota a few weeks ago, then my system gets so unresponsible, that services stopped working and nothing worked anymore... disabling qouta was also not able, it endet up in a kernel-panic and frozen system. Hat to reinstall my system back from the backup... greetings Jakob -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: send snapshot from snapshot incremental
Am 2017-03-26 um 22:07 schrieb Peter Grandi: > [ ... ] >> BUT if i take a snapshot from the system, and want to transfer >> it to the external HD, i can not set a parent subvolume, >> because there isn't any. > > Questions like this are based on incomplete understanding of > 'send' and 'receive', and on IRC user "darkling" explained it > fairly well: > >> When you use -c, you're telling the FS that it can expect to >> find a sent copy of that subvol on the receiving side, and >> that anything shared with it can be sent by reference. OK, so >> with -c on its own, you're telling the FS that "all the data >> in this subvol already exists on the remote". > >> So, when you send your subvol, *all* of the subvol's metadata >> is sent, and where that metadata refers to an extent that's >> shared with the -c subvol, the extent data isn't sent, because >> it's known to be on the other end already, and can be shared >> directly from there. > >> OK. So, with -p, there's a "base" subvol. The send subvol and >> the -p reference subvol are both snapshots of that base (at >> different times). The -p reference subvol, as with -c, is >> assumed to be on the remote FS. However, because it's known to >> be an earlier version of the same data, you can be more >> efficient in the sending by saying "start from the earlier >> version, and modify it in this way to get the new version" > >> So, with -p, not all of the metadata is sent, because you know >> you've already got most of it on the remote in the form of the >> earlier version. > >> So -p is "take this thing and apply these differences to it" >> and -c is "build this thing from scratch, but you can share >> some of the data with these sources" > For now, I think i got it... (maybe). I put the following logic into my script: 1) Search for all Subvolumes on local and remote side, where the Received-UUID on the remote side is the same as the UUID on the local side 2) Take the parent-UUID from the Snapshot i want to transfer and search in the list from 1) which snapshot (from the local side) has the same parent UUID. 3) Take the younges Snapshot from 2) ans set it as parent for the btrfs send-command 4) Search for snapshot local and remote, wich have the same name|path ans "basename" as the snapshot i want to transfer basename means, my system-subvolume is called @debian it contains one subvolume @debian/var/spool the snapshotnames are @debian_$TIMESTAMP and @debian_$TIMESTAMP/var/spool The basename is @debian and @debian/var/spool 5) set all of the snapshots with the same basename as the snapshot to be transferred as clones for btrfs send. The final command involves the youngest "sister" from the snapshot i want to transfer, which is on both sides, set as "parent", and a bunch of snapshots wich are older (or even younger - is this a problem???) than the snapshot i want to transfer wich contain modified and the same data, set as clones If there is no parent (In case of transferring a snapshot of a snapshot...) then there are clones of this snapshot, so not all of the data is to be sent again (and consumes the double space on the backup-media) If there are no parents AND clones (similar snapshots), the subvolume seems to be totally new, and the whole must be transferred. If there is a parent and clones, both of them are used to minimize the data for the transfer, and use as much as possible from the existing data/metadata on the backup-media to build the new snapshot there To use all of the similar snapshots (get by the snapshotname) as clones seems to fasten the transfer in comparison to only use the parent (this seems slower). Could this be, or is this only a "feeling? Thanks for all your advices. This helped me a lot!! regards Jakob -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: send snapshot from snapshot incremental
Thanks for that explanation. I'm sure, i didn't understand the -c option... and my english is pretty good enough for the most things I need to know in Linux-things... but not for this. :-( Am 2017-03-26 um 22:07 schrieb Peter Grandi: > [ ... ] >> BUT if i take a snapshot from the system, and want to transfer >> it to the external HD, i can not set a parent subvolume, >> because there isn't any. > > Questions like this are based on incomplete understanding of > 'send' and 'receive', and on IRC user "darkling" explained it > fairly well: > >> When you use -c, you're telling the FS that it can expect to >> find a sent copy of that subvol on the receiving side, and >> that anything shared with it can be sent by reference. OK, so >> with -c on its own, you're telling the FS that "all the data >> in this subvol already exists on the remote". > >> So, when you send your subvol, *all* of the subvol's metadata >> is sent, and where that metadata refers to an extent that's >> shared with the -c subvol, the extent data isn't sent, because >> it's known to be on the other end already, and can be shared >> directly from there. > >> OK. So, with -p, there's a "base" subvol. The send subvol and >> the -p reference subvol are both snapshots of that base (at >> different times). The -p reference subvol, as with -c, is >> assumed to be on the remote FS. However, because it's known to >> be an earlier version of the same data, you can be more >> efficient in the sending by saying "start from the earlier >> version, and modify it in this way to get the new version" > >> So, with -p, not all of the metadata is sent, because you know >> you've already got most of it on the remote in the form of the >> earlier version. > >> So -p is "take this thing and apply these differences to it" >> and -c is "build this thing from scratch, but you can share >> some of the data with these sources" > > Also here some additional details: > > http://logs.tvrrug.org.uk/logs/%23btrfs/2016-06-29.html#2016-06-29T22:39:59 > > The requirement for read-only is because in that way it is > pretty sure that the same stuff is on both origin and target > volume. > > It may help to compare with RSYNC: it has to scan both the full > origin and target trees, because it cannot be told that there is > a parent tree that is the same on origin and target; but with > option '--link-dest' it can do something similar to 'send -c'. There is Subvolume A on the send- and the receive-side. There is also Subvolume AA on the send-side from A. The parent-ID from send-AA is the ID from A. The received-ID from A on received-side A is the ID from A. To send the AA, i use the command btrfs send -p A AA|btrfs receive /path/to/receiveFS/ The received-ID from AA on received-side A is the ID from AA. Now i take a snapshot from AA on the send-side -> Called AAA If i try to send AAA to the receiving FS with btrfs send -p AA AAA|btrfs receive /path/to/receiveFS/ no parent snapshot is found. I should better take the -c Option. btrfs send -c AA AAA|btrfs receive /path/to/receiveFS/ Am I right? (Sorry, cannot test it now, i do not have my external Drive here) I might remember, that this didn't work in the past... Best regards Jakob -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
send snapshot from snapshot incremental
Hi there! I asked one or two years ago for the ability of btrfs to use btrfs send|receive with a parent-subvolume from a snapshot from a snapshot. The thing is, if i take a snapshot from my system, transfer it with btrfs send|receive to an external HD on USB-Port, make some changes to my system, which destroy my whole system, i can boot in the former taken snapshot and work from there, as if nothing happened. This part works. BUT if i take a snapshot from the system, and want to transfer it to the external HD, i can not set a parent subvolume, because there isn't any. The problem was on my last question, there are missing tansactional-numbers or something similar... I'm not a programmer, but i use btrfs very extensive and wrote a fancy python-script and a gnome-shell-extension and a fuse-filesystem to do enhanced and userfriendly snapshots an backups with btrfs - also for non-programmer users... something similar userfriendly as time-machine for apple does. And one missing feature is the described problem above... booting into an older snapshot and transfer a snapshot from this one lead to a full transfer - even if there is a transferred snapshot on the external HD. I hope, i described my problem understandable... regards Jakob -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
apt taints kernel - btrfs destroys inode
What does this mean??? Mai 01 15:36:42 aldebaran kernel: [ cut here ] Mai 01 15:36:42 aldebaran kernel: WARNING: CPU: 3 PID: 8937 at /build/linux-aGlcVo/linux-4.6~rc3/fs/btrfs/inode.c:9261 btrfs_destroy_inode+0x234/0x2a0 [btrfs] Mai 01 15:36:42 aldebaran kernel: Modules linked in: cpuid(E) udp_diag(E) tcp_diag(E) inet_diag(E) ip_set_hash_net(E) ip_set(E) nfnetlink(E) uas(E) usb_storage(E) rfcomm(E) xt_multiport(E) ctr(E) ccm(E) bnep(E) Mai 01 15:36:42 aldebaran kernel: videobuf2_v4l2(E) rtlwifi(E) joydev(E) videobuf2_core(E) mac80211(E) videodev(E) media(E) cfg80211(E) rtsx_pci_ms(E) memstick(E) battery(E) parport_pc(E) 8250_fintek(E) snd_hda Mai 01 15:36:42 aldebaran kernel: ppdev(E) lp(E) parport(E) efivarfs(E) autofs4(E) btrfs(E) crc32c_generic(E) xor(E) raid6_pq(E) sr_mod(E) cdrom(E) sd_mod(E) hid_generic(E) usbhid(E) hid(E) rtsx_pci_sdmmc(E) mm Mai 01 15:36:42 aldebaran kernel: CPU: 3 PID: 8937 Comm: apt-get Tainted: G U W E 4.6.0-rc3-amd64 #1 Debian 4.6~rc3-1~exp1 Mai 01 15:36:42 aldebaran kernel: Hardware name: Hewlett-Packard /2248, BIOS M74 Ver. 01.08 12/12/2014 Mai 01 15:36:42 aldebaran kernel: 0286 bf954d90 81310e95 Mai 01 15:36:42 aldebaran kernel: 8107a4ee 88004dd46180 88010c9f7dc8 Mai 01 15:36:42 aldebaran kernel: 8800a09eb800 88010c9f7dc8 880008b87058 88010c9f7dc8 Mai 01 15:36:42 aldebaran kernel: Call Trace: Mai 01 15:36:42 aldebaran kernel: [] ? dump_stack+0x5c/0x77 Mai 01 15:36:42 aldebaran kernel: [] ? __warn+0xbe/0xe0 Mai 01 15:36:42 aldebaran kernel: [] ? btrfs_destroy_inode+0x234/0x2a0 [btrfs] Mai 01 15:36:42 aldebaran kernel: [] ? __dentry_kill+0x178/0x1d0 Mai 01 15:36:42 aldebaran kernel: [] ? dput+0x11b/0x210 Mai 01 15:36:42 aldebaran kernel: [] ? __fput+0x164/0x1e0 Mai 01 15:36:42 aldebaran kernel: [] ? task_work_run+0x71/0x90 Mai 01 15:36:42 aldebaran kernel: [] ? exit_to_usermode_loop+0xba/0xc0 Mai 01 15:36:42 aldebaran kernel: [] ? syscall_return_slowpath+0x45/0x50 Mai 01 15:36:42 aldebaran kernel: [] ? system_call_fast_compare_end+0x94/0x96 Mai 01 15:36:42 aldebaran kernel: ---[ end trace 6c4d524799d6f2a2 ]--- -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Kernel-failure - Computer freezes
Got this Kernel-failure today while making a btrfs-snapshot. # uname -r 4.2.0-1-amd64 Dez 16 20:33:14 aldebaran kernel: btrfs: page allocation failure: order:1, mode:0x204020 Dez 16 20:33:14 aldebaran kernel: CPU: 1 PID: 8016 Comm: btrfs Tainted: G U W 4.2.0-1-amd64 #1 Debian 4.2.6-3 Dez 16 20:33:14 aldebaran kernel: Hardware name: Hewlett-Packard HP ProBook 450 G2/2248, BIOS M74 Ver. 01.08 12/12/2014 Dez 16 20:33:14 aldebaran kernel: 0001 8154f6c3 00204020 Dez 16 20:33:14 aldebaran kernel: 8115727f 88014f5fbb00 0001 Dez 16 20:33:14 aldebaran kernel: 0001 88014f5fdfa8 0046 Dez 16 20:33:14 aldebaran kernel: Call Trace: Dez 16 20:33:14 aldebaran kernel: [] ? dump_stack+0x40/0x50 Dez 16 20:33:14 aldebaran kernel: [] ? warn_alloc_failed+0xcf/0x130 Dez 16 20:33:14 aldebaran kernel: [] ? __alloc_pages_nodemask+0x2b4/0x9e0 Dez 16 20:33:14 aldebaran kernel: [] ? kmem_getpages+0x61/0x110 Dez 16 20:33:14 aldebaran kernel: [] ? fallback_alloc+0x143/0x1f0 Dez 16 20:33:14 aldebaran kernel: [] ? kmem_cache_alloc+0x1eb/0x430 Dez 16 20:33:14 aldebaran kernel: [] ? ida_pre_get+0x5c/0xd0 Dez 16 20:33:14 aldebaran kernel: [] ? get_anon_bdev+0x6d/0xe0 Dez 16 20:33:14 aldebaran kernel: [] ? btrfs_init_free_ino_ctl+0x61/0xa0 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? btrfs_init_fs_root+0x106/0x180 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? btrfs_read_fs_root+0x33/0x40 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? btrfs_get_fs_root.part.49+0x93/0x180 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? memcmp_extent_buffer+0xc1/0x120 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? btrfs_lookup_dentry+0x285/0x500 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? btrfs_lookup+0xe/0x30 [btrfs] Dez 16 20:33:14 aldebaran kernel: [] ? lookup_real+0x19/0x60 Dez 16 20:33:14 aldebaran kernel: [] ? path_openat+0xa98/0x14c0 Dez 16 20:33:14 aldebaran kernel: [] ? do_set_pte+0x9e/0xd0 Dez 16 20:33:14 aldebaran kernel: [] ? filemap_map_pages+0x21e/0x230 Dez 16 20:33:14 aldebaran kernel: [] ? do_filp_open+0x75/0xd0 Dez 16 20:33:14 aldebaran kernel: [] ? __alloc_fd+0x3f/0x110 Dez 16 20:33:14 aldebaran kernel: [] ? do_sys_open+0x12a/0x200 Dez 16 20:33:14 aldebaran kernel: [] ? system_call_fast_compare_end+0xc/0x6b Dez 16 20:33:14 aldebaran kernel: Mem-Info: Dez 16 20:33:14 aldebaran kernel: active_anon:226184 inactive_anon:65166 isolated_anon:0 active_file:245088 inactive_file:246297 isolated_file:0 unevictable:192 dirty:1706 writeback:50 unstable:0 slab_reclaimable:32362 slab_unreclaimable:16012 mapped:96648 shmem:53953 pagetables:9075 bounce:0 free:7046 free_pcp:1265 free_cma:0 Dez 16 20:33:14 aldebaran kernel: Node 0 DMA free:13364kB min:32kB low:40kB high:48kB active_anon:252kB inactive_anon:284kB active_file:560kB inactive_file:660kB unevictable:0kB isolated(anon):0kB isolated(file) Dez 16 20:33:14 aldebaran kernel: lowmem_reserve[]: 0 2125 3337 3337 Dez 16 20:33:14 aldebaran kernel: Node 0 DMA32 free:11232kB min:4624kB low:5780kB high:6936kB active_anon:612872kB inactive_anon:162128kB active_file:616016kB inactive_file:618100kB unevictable:516kB isolated(an Dez 16 20:33:14 aldebaran kernel: lowmem_reserve[]: 0 0 1212 1212 Dez 16 20:33:14 aldebaran kernel: Node 0 Normal free:3464kB min:2636kB low:3292kB high:3952kB active_anon:291612kB inactive_anon:98252kB active_file:363776kB inactive_file:366212kB unevictable:252kB isolated(ano Dez 16 20:33:14 aldebaran kernel: lowmem_reserve[]: 0 0 0 0 Dez 16 20:33:14 aldebaran kernel: Node 0 DMA: 31*4kB (UE) 17*8kB (UM) 8*16kB (UE) 5*32kB (UEM) 2*64kB (E) 1*128kB (E) 1*256kB (M) 2*512kB (EM) 1*1024kB (E) 1*2048kB (E) 2*4096kB (UM) = 13348kB Dez 16 20:33:14 aldebaran kernel: Node 0 DMA32: 2720*4kB (UEM) 30*8kB (UEM) 7*16kB (UM) 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 11232kB Dez 16 20:33:14 aldebaran kernel: Node 0 Normal: 837*4kB (UEM) 22*8kB (UM) 0*16kB 2*32kB (M) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3588kB Dez 16 20:33:14 aldebaran kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Dez 16 20:33:14 aldebaran kernel: 545434 total pagecache pages Dez 16 20:33:14 aldebaran kernel: 4 pages in swap cache Dez 16 20:33:14 aldebaran kernel: Swap cache stats: add 10, delete 6, find 0/0 Dez 16 20:33:14 aldebaran kernel: Free swap = 7812052kB Dez 16 20:33:14 aldebaran kernel: Total swap = 7812092kB Dez 16 20:33:14 aldebaran kernel: 892955 pages RAM Dez 16 20:33:14 aldebaran kernel: 0 pages HighMem/MovableOnly Dez 16 20:33:14 aldebaran kernel: 33747 pages reserved Dez 16 20:33:14 aldebaran kernel: 0 pages hwpoisoned Dez 16 20:33:14 aldebaran kernel: BUG: unable to handle kernel NULL
Re: boot from snapshot and take snapshot from here.
Am 2015-11-07 um 20:18 schrieb Hugo Mills: > On Sat, Nov 07, 2015 at 08:04:01PM +0100, Jakob Schürz wrote: >> Hi there! >> >> I can't find any Information for my Problem. And I try to explain, what >> I want. > [snip] >> BUT >> >> When i boot from f.e. @debian-snap2, the mechanics doesn't work any longer. >> >> btrfs subvol snapshot @debian-snap2 @debian-snap3 >> >> this works. >> >> btrfs send -p @debian-snap2 debian-snap3 | btrfs receive /backupmountpoint >> >> gives me an error, that btrfs can't find a subvol with a proper ID. >> >> Ho to do this, if i want to boot from a former snapshot (in case of a >> mistaken system-upgrade) and i want to transfer the later snapshots to >> the external HD - only the differences? > >Basically, you can't right now. It's a known limitation, and can, I > think, be fixed, but it'll need a change to the send stream format, > which is a tricky migration to do (because there's currently no format > identifier in it, which makes it hard to handle both formats easily). > >> There is nothing on the internet on this problem (I've found). But i >> think this is a usecase, which I'm not alone needing it... > >You're not alone. It will come eventually, but only when David's in > a position to deal with the issues of changing the send stream. OK! I'm always happy, when i realize, it's not my tired mind, when i can't find the solution :) Thanks for the info. So we will wait. jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ signature.asc Description: OpenPGP digital signature
boot from snapshot and take snapshot from here.
Hi there! I can't find any Information for my Problem. And I try to explain, what I want. There is a btrfs-partition on my laptop. I take snapshots every hour from the subvolume @debian. btrfs subvol snapshot @debian @debian-snap1 Then I transfer this snapshot to my external HD btrfs send @debian-snap1 | btrfs receive /backupmountpoint The next snapshot btrfs subvol snapshot @debian @debian-snap2 Transfer is btrfs send -p @debian-snap1 @debian-snap2|btrfs receive /backupmountpoint No magic here, works fine. BUT When i boot from f.e. @debian-snap2, the mechanics doesn't work any longer. btrfs subvol snapshot @debian-snap2 @debian-snap3 this works. btrfs send -p @debian-snap2 debian-snap3 | btrfs receive /backupmountpoint gives me an error, that btrfs can't find a subvol with a proper ID. Ho to do this, if i want to boot from a former snapshot (in case of a mistaken system-upgrade) and i want to transfer the later snapshots to the external HD - only the differences? There is nothing on the internet on this problem (I've found). But i think this is a usecase, which I'm not alone needing it... jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ signature.asc Description: OpenPGP digital signature
Re: btrfs-balance causes system-freeze on full disk
Found a new habit... I have a lot Snapshots on my drive. (Take every 10 Min a new one, every houry one, every day one, every system-update, every plugin after external HD...) If there are to much snapshots (didn't find out the count how much exactly) balancing fails with a system-freeze. if i delete some of my snapshots, balancing is working... Maybe this is a hint for bugfixing!! Do you think? greez Jakob Am 2015-10-21 um 22:51 schrieb Kyle Manna: > I had a number of similar btrfs balance crashes in the past few days, > but the disk wasn't full. You should try tailing the system logs from > a remote machine when it happens. You'll likely see some bug info > before the system dies and becomes unusable. > > The issue I encountered is described @ > https://bugzilla.kernel.org/show_bug.cgi?id=105681 > ᐧ > > On Wed, Oct 21, 2015 at 12:38 PM, Jakob Schürz > <wertsto...@nurfuerspam.de> wrote: >> Hi there! >> >> Is it possible, what i've recognized now. My system (debian) runs on >> btrfs, and i have a lot of snapshots on my hard-disk. >> Since some days my system freezes totally. I recognized, it always >> happens during btrfs-balance. >> >> So i deleted some of the old snapshots and tried another balance-run. >> Nothing happened... No system-freeze. >> >> System-freeze means: No Keyboard-action. The Mouse is frozen, the screen >> is frozen, no magic-sysreq, no ssh-login. >> >> Can btrfs cause such a freeze?? >> >> greez >> >> jakob >> -- >> http://xundeenergie.at >> http://verkehrsloesungen.wordpress.com/ >> http://cogitationum.wordpress.com/ >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ signature.asc Description: OpenPGP digital signature
Re: btrfs-balance causes system-freeze on full disk
Am 2015-10-27 um 18:09 schrieb Hugo Mills: > On Tue, Oct 27, 2015 at 05:05:55PM +0100, Jakob Schürz wrote: >> Found a new habit... >> >> I have a lot Snapshots on my drive. (Take every 10 Min a new one, every >> houry one, every day one, every system-update, every plugin after >> external HD...) >> >> If there are to much snapshots (didn't find out the count how much >> exactly) balancing fails with a system-freeze. >> if i delete some of my snapshots, balancing is working... > >The whole machine comes to a halt, or just the balancing? The whole machine. Even magic-sysreq-keys are not reacting... I have to reboot my machine using the power-button... > If you > have lots of snapshots, balance can take *insane* amounts of time -- > my big storage array, for example, takes about 60 seconds to balance a > data block group, and something like 4+ hours to balance some of the > metadata block groups. While it's "stuck" like that, it is actually > making progress, but it doesn't look like it. I know, balancint takes a huge amount of time. So my command is: /bin/btrfs balance start -dusage=55 -v When balancing is running, the machine reactes a little slower than normal... but in my case, the machine is freezing. completly and totally... :( greez jakob signature.asc Description: OpenPGP digital signature
btrfs-balance causes system-freeze on full disk
Hi there! Is it possible, what i've recognized now. My system (debian) runs on btrfs, and i have a lot of snapshots on my hard-disk. Since some days my system freezes totally. I recognized, it always happens during btrfs-balance. So i deleted some of the old snapshots and tried another balance-run. Nothing happened... No system-freeze. System-freeze means: No Keyboard-action. The Mouse is frozen, the screen is frozen, no magic-sysreq, no ssh-login. Can btrfs cause such a freeze?? greez jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs-balance causes system-freeze on full disk
Thx! It seems to be the same bug. Am 2015-10-21 um 22:51 schrieb Kyle Manna: > I had a number of similar btrfs balance crashes in the past few days, > but the disk wasn't full. You should try tailing the system logs from > a remote machine when it happens. You'll likely see some bug info > before the system dies and becomes unusable. > > The issue I encountered is described @ > https://bugzilla.kernel.org/show_bug.cgi?id=105681 > ᐧ > > On Wed, Oct 21, 2015 at 12:38 PM, Jakob Schürz > <wertsto...@nurfuerspam.de> wrote: >> Hi there! >> >> Is it possible, what i've recognized now. My system (debian) runs on >> btrfs, and i have a lot of snapshots on my hard-disk. >> Since some days my system freezes totally. I recognized, it always >> happens during btrfs-balance. >> >> So i deleted some of the old snapshots and tried another balance-run. >> Nothing happened... No system-freeze. >> >> System-freeze means: No Keyboard-action. The Mouse is frozen, the screen >> is frozen, no magic-sysreq, no ssh-login. >> >> Can btrfs cause such a freeze?? >> >> greez >> >> jakob >> -- >> http://xundeenergie.at >> http://verkehrsloesungen.wordpress.com/ >> http://cogitationum.wordpress.com/ >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Skript for backup btrfs on external HD
Am 2014-11-29 um 23:18 schrieb Marc MERLIN: On Sat, Nov 29, 2014 at 10:51:08PM +0100, Jakob Schürz wrote: Am 2014-11-29 um 22:11 schrieb Marc MERLIN: On Sat, Nov 29, 2014 at 09:34:01PM +0100, Jakob Schürz wrote: Hi there! I made a script to do backup with btrfs on a external HD. You can see the function, how it works, and how it's to be used on my site http://linux.xundeenergie.at/doku.php?id=mkbtrbackup The site is in german. An english one will follow later. Do you want some explanations? Sure, how is it different from those 3? https://btrfs.wiki.kernel.org/index.php/Incremental_Backup#Available_Backup_Tools Wheter i haven't seen it, or this scripts can't do recursive backup... That's probably right, at least not automatically. And that's why i made the script. :) If you have subvolumes in subvolumes (for example: /home, /home/user1, /home/user2 /var, /var/spool, /var/lib are extra subvolumes IN the normal filetree from linux), my script takes them all. For me, they are all subvolumes also mounted on /mnt/btrfs_poolx so I backup from there. That's also possible with my skript, because you can control it with an config-file. For example you have / |-@ |-@home `-@var And you want all your snapshots of this 3 subvolumes in separate directories with timestamp (and maybe .hourly_X-Tag) put in the config: SNPMNT=/path/to/btrfs-poolmount BKPMNT=/path/to/external/HD/mountpoint backup @ roots backup/roots backup @home homes backup/homes backup @varvarsbackup/vars start the skript with mkbtrbackup create --interval hourly -c /path/to/backupconfig you get in /path/to/btrfs-poolmount 3 directories (roots, homes and vars), and on /path/to/external/HD/mountpoint one directoriy backup, including also the three given subdirectories from the 4th coloumn (leave this coloumn blank, no auto-transfer to the external HD!!!) in this subdirectories you get subvolumes like @.20141130-115001.hourly_0 @home.20141130-115001.hourly_0 @var.20141130-115001.hourly_0 AND they are rotated automatically. And my script changes the fstab-entry in the new snapshot. The original has the option subvol=@SUBVOL, where @SUBVOL is the name of the original system. I don't need to do that, my script updates a symlink pointing to the last snapshot, and you can use subvol=symlink-name I'm trying on this, it's not finished. There are many discussions about. What is better... modify grub.cfg on each snapshot, work with symlinks... I create one symlink @*.CURRENT. I will rename it to .LAST... so i can do the same with a static grub-entry You get a systemd-unit, in the tarball, which makes a snapshot from your system, on successful boot, so you can switch back fast, if an update destroyed your system. And it is for minimal-systems... no python, no perl, no java... only shell(bash) :-) That makes sense, thanks for explaining. For example... on an raspberry Pi it would be a good thing. :) Hope, you try it, and give me some feedback. ;-) Jakob -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Skript for backup btrfs on external HD
Am 2014-11-29 um 22:11 schrieb Marc MERLIN: On Sat, Nov 29, 2014 at 09:34:01PM +0100, Jakob Schürz wrote: Hi there! I made a script to do backup with btrfs on a external HD. You can see the function, how it works, and how it's to be used on my site http://linux.xundeenergie.at/doku.php?id=mkbtrbackup The site is in german. An english one will follow later. Do you want some explanations? Sure, how is it different from those 3? https://btrfs.wiki.kernel.org/index.php/Incremental_Backup#Available_Backup_Tools Wheter i haven't seen it, or this scripts can't do recursive backup... If you have subvolumes in subvolumes (for example: /home, /home/user1, /home/user2 /var, /var/spool, /var/lib are extra subvolumes IN the normal filetree from linux), my script takes them all. It looks on the external storage, if there's an older snapshot (i call all subvolumes together in this case a snapshot!!) which is also on the local machine. If so, is makes a incremental backup. If not, a initial transfer is started. For each subvolume in the snapshot! And my script changes the fstab-entry in the new snapshot. The original has the option subvol=@SUBVOL, where @SUBVOL is the name of the original system. It changes the @SUBVOLUME to the subvolume-id, so you can mount your snapshot easy. One Point is missing... Modifying of grub to serve boot-menu-entries for older snapshots. You get a systemd-unit, in the tarball, which makes a snapshot from your system, on successful boot, so you can switch back fast, if an update destroyed your system. And it is for minimal-systems... no python, no perl, no java... only shell(bash) :-) regards jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Skript for backup btrfs on external HD
Hi there! I made a script to do backup with btrfs on a external HD. You can see the function, how it works, and how it's to be used on my site http://linux.xundeenergie.at/doku.php?id=mkbtrbackup The site is in german. An english one will follow later. Do you want some explanations? greetings jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
systemd.setenv and a mount.unit
Hi there! Another challenge... I'm using btrfs. So i make snapshots from my system. And in a script, I make a symlink (for example: @system.CURRENT and @system.LAST) for the current and the last snapshot. So i want to add 2 entries in grub2 from which i can boot into the current and the last snapshot. I tried to pass an environmental variable with systemd.setenv=BOOTSNAP=@system.CURRENT, and i have a mount-unit containing the option Options=defaults,nofail,subvol=archive-local/@system.$BOOTSNAP but it doesn't work. If i change $BOOTSNAP to CURRENT, the mount works. So i made a test.service, containing only StartExec=/bin/echo $BOOTSNAP I get the value @system.CURRENT in the logs... How can I do this mounts? jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: systemd.setenv and a mount.unit
Am 2014-11-20 um 11:17 schrieb Goffredo Baroncelli: On 2014-11-19 23:48, Jakob Schürz wrote: Hi there! Another challenge... I'm using btrfs. So i make snapshots from my system. And in a script, I make a symlink (for example: @system.CURRENT and @system.LAST) for the current and the last snapshot. Interesting, I was unaware that I could mount a subvolume passing a soft link. Fortunately Yes, that works. :) So i came on this idea, to create 2 (or more) stable links on the current, last and so on snapshot. The Problem is the fstab... So i want to add 2 entries in grub2 from which i can boot into the current and the last snapshot. I tried to pass an environmental variable with systemd.setenv=BOOTSNAP=@system.CURRENT, and i have a mount-unit containing the option Options=defaults,nofail,subvol=archive-local/@system.$BOOTSNAP but it doesn't work. If i change $BOOTSNAP to CURRENT, the mount works. So i made a test.service, containing only StartExec=/bin/echo $BOOTSNAP I get the value @system.CURRENT in the logs... This is more a systemd related question. However it seems that ExecStart supports ...basic environment variable substitution...; but the mount unit doesn't. This explain your difficulties. Anyway I suggest you to contact the systemd developers to get further support; maybe that this could be a add as TODO item. I hope so... and i found out, that this is not a btrfs-challenge. On the grub-devel-list i saw a discussion to make dynamically entries for all snapshots in a certain directory according to this challenge. But this also didn't solve the fstab-challenge. So i think i have to place it on a systemd-devel-list... How can I do this mounts? For the boot, I used the rootflags= command line options. This usually is interpreted by the initrd/initramfs as option to pass to the mount command. In my case I have: rootflags=subvol=debian so, the subvol=debian option is passed to mount. When grub-mkconfig generates the grub menu entries, does so. This I also have in my grub-config, and it works. But it doesn't solve the challenge. Mounting more subvolumes according to the root-subvolume... :-) regards Jakob -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs send and an existing backup
Hi there! I'm new on btrfs, and I like it :) But i have a question. I have a existing backup on an external HDD. This was ext4 before i converted it to btrfs. And i installed my debian new on btrfs with some subvolumes. (f.e. home, var, multimedia/Video multimedia/Audio...) On my backup there are no subvolumes. Now i wrote a script to take local snapshots on my laptops HDD an mirror this snapshots with btrfs send/receive to the external HDD. An i don't know, how to do, to make the inital snapshot on the external HDD. I want to use the existing data there, so I don't have to transmit the whole bunch of data to the external drive, which exists there already... What happens, if i make the same structure on the external drive with creating subvolumes and »cp --reflink«, give this subvolumes the correct names, and fire a »btrfs send«? Or is the best (ONLY???) way, to make an initial snapshot on the external drive and delete the old backup there? greetings jakob -- http://xundeenergie.at http://verkehrsloesungen.wordpress.com/ http://cogitationum.wordpress.com/ -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
