Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
On 06/04/2017 16:31, Gionatan Danti wrote: Hi all, I'm seeking some advice for a new virtualization system (KVM) on top of LVM. The goal is to take agentless backups via LVM snapshots. In short: what you suggest to snapshot a quite big (8+ TB) volume? Classic LVM (with old snapshot behavior) or thinlvm (and its new snapshot method)? Long story: In the past, I used classical, preallocated logical volumes directly exported as virtual disks. In this case, I snapshot the single LV I want to backup and, using dd/ddrescue, I copy it. Problem is this solution prevents any use of thin allocation or sparse files, so I tried to replace it with something filesystem-based. Lately I used another approach, configuring a single thinly provisioned LV (with no zeroing) + XFS + raw or qcow2 virtual machine images. To make backups, I snapshotted the entire thin LV and, after mounting it, I copied the required files. So far this second solution worked quite well. However, before using it in more and more installations, I wonder if it is the correct approach or if something better, especially from a stability standpoint, is possible. Gived that I would like to use XFS, and that I need snapshots at the block level, two possibilities came to mind: 1) continue to use thinlvm + thin snapshots + XFS. What do you think about a 8+ TB thin pool/volume with relatively small (64/128KB) chunks? Would you be comfortable using it in production workloads? What about powerloss protection? From my understanding, thinlvm passes flushes anytime the higher layers issue them and so should be reasonable safe against unexpected powerloss. Is this view right? 2) use a classic (non-thin) LVM + normal snapshot + XFS. I know for sure that LV size is not an issue here, however big snapshot size used to be problematic: the CoW table had to be read completely before the snapshot can be activated. Is this problem a solved one? Or big snapshot can be problematic? Thank you all. Hi, anyone with other thoughts on the matter? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.da...@assyoma.it - i...@assyoma.it GPG public key ID: FF5F32A8 ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
Gionatan Danti schreef op 13-04-2017 12:20: Hi, anyone with other thoughts on the matter? I wondered why a single thin LV does work for you in terms of not wasting space or being able to make more efficient use of "volumes" or client volumes or whatever. But a multitude of thin volumes won't. See, you only compared multiple non-thin with a single-thin. So my question is: did you consider multiple thin volumes? ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
Using a classic snapshot for backup does not normally involve activating a large CoW. I generally create a smallish snapshot (a few gigs), that will not fill up during the backup process. If for some reason, a snapshot were to fill up before backup completion, reads from the snapshot get I/O errors (I've tested this), which alarms and aborts the backup. Yes, keeping a snapshot around and activating it at boot can be a problem as the CoW gets large. If you are going to keep snapshots around indefinitely, the thinpools are probably the way to go. (What happens when you fill up those? Hopefully it "freezes" the pool rather than losing everything.) On 04/07/2017 12:33 PM, Gionatan Danti wrote: > For the logical volume itself, I target a 8+ TB size. However, what > worries me is *not* LV size by itself (I know that LVM can be used on > volume much bigger than that), rather the snapshot CoW table. In > short, from reading this list and from first-hand testing, big > snapshots (20+ GB) require lenghtly activation, due to inefficiency in > how classic metadata (ie: non thinly-provided) are layed out/used. > However, I read that this was somewhat addressed lately. Do you have > any insight? > ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
Stuart Gathman schreef op 13-04-2017 14:59: If you are going to keep snapshots around indefinitely, the thinpools are probably the way to go. (What happens when you fill up those? Hopefully it "freezes" the pool rather than losing everything.) My experience is that the system crashes. I have not tested this with a snapshot but a general thin pool overflow crashes the system. Within half a minute, I think. It is irrelevant whether the volumes had anything to do with the operation of the system; ie. some mounted volumes that you write to that are in no other use will crash the system. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
Dne 13.4.2017 v 15:52 Xen napsal(a): Stuart Gathman schreef op 13-04-2017 14:59: If you are going to keep snapshots around indefinitely, the thinpools are probably the way to go. (What happens when you fill up those? Hopefully it "freezes" the pool rather than losing everything.) My experience is that the system crashes. I have not tested this with a snapshot but a general thin pool overflow crashes the system. Within half a minute, I think. It is irrelevant whether the volumes had anything to do with the operation of the system; ie. some mounted volumes that you write to that are in no other use will crash the system. Hello Just let's repeat. Full thin-pool is NOT in any way comparable to full filesystem. Full filesystem has ALWAYS room for its metadata - it's not pretending it's bigger - it has 'finite' space and expect this space to just BE there. Now when you have thin-pool - it cause quite a lot of trouble across number of layers. There are solvable and being fixed. But as the rule #1 still applies - do not run your thin-pool out of space - it will not always heal easily without losing date - there is not a simple straighforward way how to fix it (especially when user cannot ADD any new space he promised to have) So monitoring pool and taking action ahead in time is always superior solution to any later postmortem systems restores. Regards Zdenek ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
Zdenek Kabelac schreef op 13-04-2017 16:33: Hello Just let's repeat. Full thin-pool is NOT in any way comparable to full filesystem. Full filesystem has ALWAYS room for its metadata - it's not pretending it's bigger - it has 'finite' space and expect this space to just BE there. Now when you have thin-pool - it cause quite a lot of trouble across number of layers. There are solvable and being fixed. But as the rule #1 still applies - do not run your thin-pool out of space - it will not always heal easily without losing date - there is not a simple straighforward way how to fix it (especially when user cannot ADD any new space he promised to have) So monitoring pool and taking action ahead in time is always superior solution to any later postmortem systems restores. Yes that's what I said. If your thin pool runs out, your system will crash. Thanks for alluding that this will also happen if a thin snapshot causes this (obviously). Regards. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
On 04/13/2017 10:33 AM, Zdenek Kabelac wrote: > > > Now when you have thin-pool - it cause quite a lot of trouble across > number of layers. There are solvable and being fixed. > > But as the rule #1 still applies - do not run your thin-pool out of > space - it will not always heal easily without losing date - there is > not a simple straighforward way how to fix it (especially when user > cannot ADD any new space he promised to have) IMO, the friendliest thing to do is to freeze the pool in read-only mode just before running out of metadata. While still involving application level data loss (the data it was just trying to write), and still crashing the system (the system may be up and pingable and maybe even sshable, but is "crashed" for normal purposes), it is simple to understand and recover. A sysadmin could have a plain LV for the system volume, so that logs and stuff would still be kept, and admin logins work normally. There is no panic, as the data is there read-only. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
Stuart Gathman schreef op 13-04-2017 17:29: IMO, the friendliest thing to do is to freeze the pool in read-only mode just before running out of metadata. It's not about metadata but about physical extents. In the thin pool. While still involving application level data loss (the data it was just trying to write), and still crashing the system (the system may be up and pingable and maybe even sshable, but is "crashed" for normal purposes) Then it's not crashed. Only some application that may make use of the data volume may be crashed, but not the entire system. The point is that errors and some filesystem that has errors=remount-ro, is okay. If a regular snapshot that is mounted fills up, the mount is dropped. System continues operating, as normal. , it is simple to understand and recover. A sysadmin could have a plain LV for the system volume, so that logs and stuff would still be kept, and admin logins work normally. There is no panic, as the data is there read-only. Yeah a system panic in terms of some volume becoming read-only is perfectly acceptable. However the kernel going entirely mayhem, is not. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
On Thu, 13 Apr 2017, Xen wrote: Stuart Gathman schreef op 13-04-2017 17:29: understand and recover. A sysadmin could have a plain LV for the system volume, so that logs and stuff would still be kept, and admin logins work normally. There is no panic, as the data is there read-only. Yeah a system panic in terms of some volume becoming read-only is perfectly acceptable. However the kernel going entirely mayhem, is not. Heh. I was actually referring to *sysadmin* panic, not kernel panic. :-) But yeah, sysadmin panic can result in massive data loss... -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] Snapshot behavior on classic LVM vs ThinLVM
On Thu, 13 Apr 2017, Xen wrote: Stuart Gathman schreef op 13-04-2017 17:29: IMO, the friendliest thing to do is to freeze the pool in read-only mode just before running out of metadata. It's not about metadata but about physical extents. In the thin pool. Ok. My understanding is that *all* the volumes in the same thin-pool would have to be frozen when running out of extents, as writes all pull from the same pool of physical extents. -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
