Re: Btrfs by default, the compression option

2020-08-01 Thread Chris Murphy
On Sat, Aug 1, 2020 at 6:43 AM Artem Tim wrote: > > @Chris, thanks, i did some tests and now we have some numbers. tl;rd the > results is pretty satisfying me, even if there no speedup in my case, but > there is a lot disk space could be saved and reduce writes. Probably not > worth file a

Re: Btrfs by default, the compression option

2020-08-01 Thread Artem Tim
@Chris, thanks, i did some tests and now we have some numbers. tl;rd the results is pretty satisfying me, even if there no speedup in my case, but there is a lot disk space could be saved and reduce writes. Probably not worth file a bug, but a little bit weird that with zstd:1 still a little

Re: Btrfs by default, the compression option

2020-07-29 Thread Samuel Sieb
On 7/10/20 1:56 AM, Nicolas Mailhot via devel wrote: Le jeudi 09 juillet 2020 à 23:47 +, Zachary Lym a écrit : Yes, it's completely reasonable to not do it. It might seem like a big change on its own, but Btrfs has had native compression for 10+ years, and at least three years for most all

Re: Btrfs by default, the compression option

2020-07-29 Thread Chris Murphy
On Wed, Jul 29, 2020 at 2:59 PM Artem Tim wrote: > > I am experimenting now with various compression in Fedora and noticed > slowdowns with compression only for mock at this moment. On 4-core CPU > building in mock with zstd:1 on HDD is slower. Also 'autodefrag' and > 'space_cache=v2' options

Re: Btrfs by default, the compression option

2020-07-29 Thread Chris Murphy
On Wed, Jul 29, 2020 at 2:59 PM Artem Tim wrote: > > I am experimenting now with various compression in Fedora and noticed > slowdowns with compression only for mock at this moment. On 4-core CPU > building in mock with zstd:1 on HDD is slower. Also 'autodefrag' and > 'space_cache=v2' options

Re: Btrfs by default, the compression option

2020-07-29 Thread Artem Tim
I am experimenting now with various compression in Fedora and noticed slowdowns with compression only for mock at this moment. On 4-core CPU building in mock with zstd:1 on HDD is slower. Also 'autodefrag' and 'space_cache=v2' options enabled on HDD.

Re: Btrfs by default, the compression option

2020-07-10 Thread Chris Murphy
> https://paste.centos.org/view/f4165396 > > Two workloads: install and update. It might seem like an update is > both read and write dependent, but the rpms are already compressed and > don't get compressed again. The differences, I expect, are mostly > write performance. And this suggests it's

Re: Btrfs by default, the compression option

2020-07-10 Thread Mark Otaris
Maybe Workstation could provide Déjà Dup in the default system to make it discoverable and encourage users to create backups. It has to be an installed application (can’t be a website), and most users would benefit from having backups. ___ devel mailing

Re: Btrfs by default, the compression option

2020-07-10 Thread Chris Murphy
On Fri, Jul 10, 2020, 8:37 AM Matthew Miller wrote: > On Fri, Jul 10, 2020 at 12:59:37AM -0600, Chris Murphy wrote: > > https://paste.centos.org/view/f4165396 > > > > Two workloads: install and update. It might seem like an update is > > both read and write dependent, but the rpms are already

Re: Btrfs by default, the compression option

2020-07-10 Thread Matthew Miller
On Fri, Jul 10, 2020 at 12:59:37AM -0600, Chris Murphy wrote: > https://paste.centos.org/view/f4165396 > > Two workloads: install and update. It might seem like an update is > both read and write dependent, but the rpms are already compressed and > don't get compressed again. The differences, I

Re: Btrfs by default, the compression option

2020-07-10 Thread Nicolas Mailhot via devel
Le jeudi 09 juillet 2020 à 23:47 +, Zachary Lym a écrit : > > Yes, it's completely reasonable to not do it. It might seem like a > > big > > change on its own, but Btrfs has had native compression for 10+ > > years, > > and at least three years for most all of the workloads at Facebook. > > So

Re: Btrfs by default, the compression option

2020-07-10 Thread Chris Murphy
On Wed, Jul 8, 2020 at 9:50 AM Matthew Miller wrote: > > More data is always better. I like qualifying the situations in that way. I > think we should make our decision based on the "center" rather than the > edges, though. > > For I hope obvious reasons, I'd love to see this tested on a Lenovo

Re: Btrfs by default, the compression option

2020-07-09 Thread drago01
On Friday, July 10, 2020, Zachary Lym wrote: > > Yes, it's completely reasonable to not do it. It might seem like a big > > change on its own, but Btrfs has had native compression for 10+ years, > > and at least three years for most all of the workloads at Facebook. So > > it's quite safe. > >

Re: Btrfs by default, the compression option

2020-07-09 Thread Zachary Lym
> Yes, it's completely reasonable to not do it. It might seem like a big > change on its own, but Btrfs has had native compression for 10+ years, > and at least three years for most all of the workloads at Facebook. So > it's quite safe. But it has been eating data as recently as 2018 [1] and the

Re: Btrfs by default, the compression option

2020-07-09 Thread Matthew Miller
On Wed, Jul 08, 2020 at 02:20:06PM -0700, John M. Harris Jr wrote: > > Yeah I guess for /usr the most relevant write metric is "does it slow down > > DNF upgrades or install operations enough to be noticeable / annoying / > > problematic"? > More importantly, does it hurt the performance of

Re: Btrfs by default, the compression option

2020-07-08 Thread John M. Harris Jr
On Wednesday, July 8, 2020 10:20:30 AM MST Matthew Miller wrote: > On Wed, Jul 08, 2020 at 11:08:53AM -0600, Chris Murphy wrote: > > > I expect beefy CPU systems, including gaming systems, to have the same > > or better read/write performance using mount option compress=zstd:1. > > Where I've

Re: Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
On Wed, Jul 8, 2020 at 12:54 PM Adam Williamson wrote: > > On Wed, 2020-07-08 at 00:24 -0600, Chris Murphy wrote: > > Hi, > > > > The change proposal has a 'compression option' and we kinda need to > > get organized. > > https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Compression > > It

Re: Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
On Wed, Jul 8, 2020 at 1:45 PM Matthew Miller wrote: > > On Wed, Jul 08, 2020 at 11:37:11AM -0600, Chris Murphy wrote: > > If it's the center, I think that favors the mount option approach and > > do it with the lowest level of compression, i.e. zstd:1. But this > > suggests more benchmarking

Re: Btrfs by default, the compression option

2020-07-08 Thread Matthew Miller
On Wed, Jul 08, 2020 at 11:37:11AM -0600, Chris Murphy wrote: > If it's the center, I think that favors the mount option approach and > do it with the lowest level of compression, i.e. zstd:1. But this > suggests more benchmarking still, to make certain it's well understood > what the range of

Re: Btrfs by default, the compression option

2020-07-08 Thread Martin Jackson
It feels to me like this might be a great area to slow down a bit and not try to do everything at once. Why don't we just make the simplest change for F33 - going to btrfs by default - and see how that goes, and consider the 'options' for F34 or later, rather than changing too much stuff at

Re: Btrfs by default, the compression option

2020-07-08 Thread Adam Williamson
On Wed, 2020-07-08 at 00:24 -0600, Chris Murphy wrote: > Hi, > > The change proposal has a 'compression option' and we kinda need to > get organized. > https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Compression It feels to me like this might be a great area to slow down a bit and not try

Re: Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
On Wed, Jul 8, 2020 at 11:20 AM Matthew Miller wrote: > > On Wed, Jul 08, 2020 at 11:08:53AM -0600, Chris Murphy wrote: > > I expect beefy CPU systems, including gaming systems, to have the same > > or better read/write performance using mount option compress=zstd:1. > > Where I've seen equal or

Re: Btrfs by default, the compression option

2020-07-08 Thread Neal Gompa
On Wed, Jul 8, 2020 at 1:20 PM Matthew Miller wrote: > > On Wed, Jul 08, 2020 at 11:08:53AM -0600, Chris Murphy wrote: > > I expect beefy CPU systems, including gaming systems, to have the same > > or better read/write performance using mount option compress=zstd:1. > > Where I've seen equal or

Re: [External] Re: Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
On Wed, Jul 8, 2020 at 10:54 AM Mark Pearson wrote: > > I personally haven't had any experience with btrfs but if there are any > guidelines on testing that we can do and what data should be collected to > help out let me know and I'll see if we can hit up some of our platforms and > get some

Re: Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
On Wed, Jul 8, 2020 at 9:50 AM Matthew Miller wrote: > > On Wed, Jul 08, 2020 at 12:24:27AM -0600, Chris Murphy wrote: > > 2. Benchmarking: this is hard. A simple tool for doing comparisons > > among algorithms on a specific bit of hardware is lzbench. > > https://github.com/inikep/lzbench > >

Re: Btrfs by default, the compression option

2020-07-08 Thread Matthew Miller
On Wed, Jul 08, 2020 at 11:08:53AM -0600, Chris Murphy wrote: > I expect beefy CPU systems, including gaming systems, to have the same > or better read/write performance using mount option compress=zstd:1. > Where I've seen equal or better read performance, there can be a write > performance drop

Re: Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
On Wed, Jul 8, 2020 at 5:13 AM Kamil Paral wrote: > > On Wed, Jul 8, 2020 at 11:22 AM Dominique Martinet > wrote: >> >> Please test, but if a file is deemed not compressible (based on, not >> sure what? the first few blocks?) then it will be stored in the >> non-compressed version. >> You can

RE: [External] Re: Btrfs by default, the compression option

2020-07-08 Thread Mark Pearson
> -Original Message- > From: Matthew Miller > Sent: Wednesday, July 8, 2020 11:51 AM > > On Wed, Jul 08, 2020 at 12:24:27AM -0600, Chris Murphy wrote: > > 2. Benchmarking: this is hard. A simple tool for doing comparisons > > among algorithms on a specific bit of hardware is lzbench. > >

Re: Btrfs by default, the compression option

2020-07-08 Thread Matthew Miller
On Wed, Jul 08, 2020 at 12:24:27AM -0600, Chris Murphy wrote: > 2. Benchmarking: this is hard. A simple tool for doing comparisons > among algorithms on a specific bit of hardware is lzbench. > https://github.com/inikep/lzbench > How to compile on F32. > https://github.com/inikep/lzbench/issues/69

Re: Btrfs by default, the compression option

2020-07-08 Thread John M. Harris Jr
On Wednesday, July 8, 2020 1:10:12 AM MST Kamil Paral wrote: > On Wed, Jul 8, 2020 at 8:26 AM Chris Murphy wrote: > > D. Which directories? Some may be outside of the installer's scope. > > > > /usr > > /var/lib/flatpak > > ~/.local/share/flatpak > > I have a concern regarding games. Currently,

Re: Btrfs by default, the compression option

2020-07-08 Thread Kamil Paral
On Wed, Jul 8, 2020 at 11:22 AM Dominique Martinet wrote: > Please test, but if a file is deemed not compressible (based on, not > sure what? the first few blocks?) then it will be stored in the > non-compressed version. > You can check with compsize after the fact if the file had been >

Re: Btrfs by default, the compression option

2020-07-08 Thread Dominique Martinet
Kamil Paral wrote on Wed, Jul 08, 2020: > On Wed, Jul 8, 2020 at 8:26 AM Chris Murphy wrote: > > > D. Which directories? Some may be outside of the installer's scope. > > > > /usr > > /var/lib/flatpak > > ~/.local/share/flatpak > > I have a concern regarding games. Currently, we have a few a

Re: Btrfs by default, the compression option

2020-07-08 Thread Kamil Paral
On Wed, Jul 8, 2020 at 8:26 AM Chris Murphy wrote: > D. Which directories? Some may be outside of the installer's scope. > > /usr > /var/lib/flatpak > ~/.local/share/flatpak > I have a concern regarding games. Currently, we have a few a bit more demanding titles on Flathub already, like 0AD,

Btrfs by default, the compression option

2020-07-08 Thread Chris Murphy
Hi, The change proposal has a 'compression option' and we kinda need to get organized. https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Compression - Compression saves space, significantly reduces write amplification and therefore increases flash lifespan, and in some cases increases