Re: number of subvolumes

2017-09-01 Thread Austin S. Hemmelgarn
On 2017-09-01 06:21, ein wrote: Very comprehensive, thank you. I was asking because I'd like to learn how really random writes by VM affects BTRFS (vs XFS,Ext4) performance and try to develop some workaround to reduce/prevent it while having csums, cow (snapshots) and compression. I've

Re: number of subvolumes

2017-09-01 Thread ein
On 08/31/2017 06:18 PM, Duncan wrote: [...] > Michał Sokołowski posted on Thu, 31 Aug 2017 16:38:14 +0200 as excerpted: >> Is there another tool to verify fragments number of given file when >> using compression? > AFAIK there isn't an official one, tho someone posted a script (python, > IIRC) at

Re: number of subvolumes

2017-08-31 Thread Duncan
Michał Sokołowski posted on Thu, 31 Aug 2017 16:38:14 +0200 as excerpted: > On 08/31/2017 01:18 PM, Austin S. Hemmelgarn wrote: >> [...] >>> Any hint here? >> Having compression enabled causes no issues with defray and balance. >> There appears to be a prevalent belief however that defrag is

Re: number of subvolumes

2017-08-31 Thread Michał Sokołowski
On 08/31/2017 01:18 PM, Austin S. Hemmelgarn wrote: > [...] >> Any hint here? > Having compression enabled causes no issues with defray and balance. > There appears to be a prevalent belief however that defrag is > pointless if you're using compression, probably because some versions > of

Re: number of subvolumes

2017-08-31 Thread Austin S. Hemmelgarn
On 2017-08-31 02:49, Ulli Horlacher wrote: On Thu 2017-08-24 (18:45), Peter Grandi wrote: As usual with Btrfs, there are corner cases to avoid: 'defrag' should be done before 'balance' Good hint. So far I did it the other way: balance before defrag. I will switch. For reference, the reason

Re: number of subvolumes

2017-08-31 Thread Ulli Horlacher
On Thu 2017-08-24 (18:45), Peter Grandi wrote: > As usual with Btrfs, there are corner cases to avoid: 'defrag' > should be done before 'balance' Good hint. So far I did it the other way: balance before defrag. I will switch. > and with compression switched off I have filesystems with

Re: number of subvolumes

2017-08-25 Thread Austin S. Hemmelgarn
On 2017-08-25 08:55, Ferry Toth wrote: Op Fri, 25 Aug 2017 07:45:44 -0400, schreef Austin S. Hemmelgarn: On 2017-08-24 17:56, Ferry Toth wrote: Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: We find that typically apt is very slow on a machine with 50 or so snapshots and raid10.

Re: number of subvolumes

2017-08-25 Thread Ferry Toth
Op Fri, 25 Aug 2017 07:45:44 -0400, schreef Austin S. Hemmelgarn: > On 2017-08-24 17:56, Ferry Toth wrote: >> Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: >> We find that typically apt is very slow on a machine with 50 or so snapshots and raid10. Slow as in probably 10x

Re: number of subvolumes

2017-08-25 Thread Austin S. Hemmelgarn
On 2017-08-24 17:56, Ferry Toth wrote: Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: We find that typically apt is very slow on a machine with 50 or so snapshots and raid10. Slow as in probably 10x slower as doing the same update on a machine with 'single' and no snapshots. Other

Re: number of subvolumes

2017-08-24 Thread Chris Murphy
On Thu, Aug 24, 2017 at 3:56 PM, Ferry Toth wrote: > Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: > >>> We find that typically apt is very slow on a machine with 50 or so >>> snapshots and raid10. Slow as in probably 10x slower as doing the same >>> update on a

Re: number of subvolumes

2017-08-24 Thread Ferry Toth
Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: >> We find that typically apt is very slow on a machine with 50 or so >> snapshots and raid10. Slow as in probably 10x slower as doing the same >> update on a machine with 'single' and no snapshots. >> >> Other operations seem to be the

Re: number of subvolumes

2017-08-24 Thread Marat Khalili
> We find that typically apt is very slow on a machine with 50 or so snapshots > and raid10. Slow as in probably 10x slower as doing the same update on a > machine with 'single' and no snapshots. > > Other operations seem to be the same speed, especially disk benchmarks do not > seem to

Re: number of subvolumes

2017-08-24 Thread Peter Grandi
>> Using hundreds or thousands of snapshots is probably fine >> mostly. As I mentioned previously, with a link to the relevant email describing the details, the real issue is reflinks/backrefs. Usually subvolume and snapshots involve them. > We find that typically apt is very slow on a machine

Re: number of subvolumes

2017-08-23 Thread Ferry Toth
Op Wed, 23 Aug 2017 10:37:07 +0200, schreef A L: > From: Ulli Horlacher -- Sent: > 2017-08-23 - 09:18 > >> On Tue 2017-08-22 (22:48), Ulli Horlacher wrote: >> >>> > Assumptions that all Btrfs features such as snapshots are infinitely >>> > scalable at

Re: number of subvolumes

2017-08-23 Thread Peter Grandi
> This is a vanilla SLES12 installation: [ ... ] Why does SUSE > ignore this "not too many subvolumes" warning? As in many cases with Btrfs "it's complicated" because of the interaction of advanced features among themselves and the chosen implementation and properties of storage; anisotropy

Re: number of subvolumes

2017-08-23 Thread A L
From: Ulli Horlacher -- Sent: 2017-08-23 - 09:18 > On Tue 2017-08-22 (22:48), Ulli Horlacher wrote: > >> > Assumptions that all Btrfs features such as snapshots are >> > infinitely scalable at no cost may be optimistic: >> > >> > >> >

number of subvolumes

2017-08-23 Thread Ulli Horlacher
On Tue 2017-08-22 (22:48), Ulli Horlacher wrote: > > Assumptions that all Btrfs features such as snapshots are > > infinitely scalable at no cost may be optimistic: > > > > > > https://btrfs.wiki.kernel.org/index.php/Gotchas#Having_many_subvolumes_can_be_very_slow > > "when you do device