Hello,
I have implemented the necessary changes in Pungi, the software that
creates Fedora compose. So this change has a potential of moving forward.
I have created 2 pull-requests for release engineering for this change:
https://pagure.io/pungi-fedora/pull-request/871
On Wed, May 13, 2020 at 6:03 PM Chris Murphy wrote:
>
> On Wed, May 13, 2020 at 5:32 AM Bohdan Khomutskyi wrote:
> >
> > Hello,
> >
> > It was a long time since the last message in this change proposal.
> >
> > Recently I was working to reduce the impact of the increased compression
> > ratio
On Wed, May 13, 2020 at 13:32:19 +0200,
Bohdan Khomutskyi wrote:
For optimization of the SquashFS, I will work on requesting the
support of the required functionality in the Pungi compose build
software.
Note that squashfs-tools 4.4 just went into rawhide a couple of days ago.
By default
On Wed, May 13, 2020 at 5:32 AM Bohdan Khomutskyi wrote:
>
> Hello,
>
> It was a long time since the last message in this change proposal.
>
> Recently I was working to reduce the impact of the increased compression
> ratio on the installation image size for Fedora. I have achieved outstanding
On Wed, May 13, 2020 at 1:33 PM Bohdan Khomutskyi
wrote:
> Hello,
>
> It was a long time since the last message in this change proposal.
>
> Recently I was working to reduce the impact of the increased compression
> ratio on the installation image size for Fedora. I have achieved
> outstanding
Hello,
It was a long time since the last message in this change proposal.
Recently I was working to reduce the impact of the increased compression
ratio on the installation image size for Fedora. I have achieved
outstanding results -- working proof of concept. With the following
change:
Hello,
I opened a request to anaconda team with a draft patch to use unsquashfs
instead of rsync: https://github.com/rhinstaller/anaconda/pull/2292.
That should lower the installation time from Live media.
Adjustments should be made to make this patch work, I was not able to
install the image
On Mon, Jan 20, 2020 at 1:38 AM Bohdan Khomutskyi wrote:
>
> In my previous message, I mentioned that CPU is underutilized during
> installation. I haven't investigated further why, but I suspect it's due to
> the inefficiency caused by the usage of the loop device and/or inefficiency
> in the
On Mon, Jan 20, 2020 at 1:38 AM Bohdan Khomutskyi wrote:
> In my previous message, I mentioned that CPU is underutilized during
> installation. I haven't investigated further why, but I suspect it's due to
> the inefficiency caused by the usage of the loop device and/or inefficiency
> in the
On Sunday, January 19, 2020 4:41:06 PM MST Chris Murphy wrote:
> I admit I'm biased toward the two endpoints: create and consume, not
> distribution ,i.e the mirror donors. Their storage and bandwidth
> concerns were evaluated with the RPM change from xz to zstd. So I'm
> mystified by the bias for
On Mon, Jan 20, 2020 at 4:24 AM Adam Williamson
wrote:
>
> On Sun, 2020-01-12 at 15:02 -0700, Chris Murphy wrote:
> > Do you have any tests to compare plain squashfs xz with zstd? The nested
> > ext4 stuff is really pointless now because Fedora hasn't used 'dd' +
> > resizing the ext4 file system
On Sun, Jan 19, 2020 at 4:42 PM Bohdan Khomutskyi
wrote:
> Hello,
>
> Thanks everyone for posting feedback.
> More benchmarking results are available at
> https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS,
> including the 'plain' SquashFS filesystem.
> After performing the tests,
On Sun, 2020-01-12 at 15:02 -0700, Chris Murphy wrote:
> Do you have any tests to compare plain squashfs xz with zstd? The nested
> ext4 stuff is really pointless now because Fedora hasn't used 'dd' +
> resizing the ext4 file system as an installation method in a long time
> (going back to Fedora
On Mon, Jan 20, 2020 at 09:37:32AM +0100, Bohdan Khomutskyi wrote:
> Chris,
>
> Thanks for your feedback and comments, it's very valuable to me.
>
> In my previous message, I mentioned that CPU is *underutilized* during
> installation. I haven't investigated further why, but I suspect it's due
On Sun, Jan 05, 2020 at 10:08:07AM -0700, Chris Murphy wrote:
> In my testing, xz does provide better compression ratios, well suited
> for seldom used images like archives. But it really makes the
> installation experience worse by soaking the CPU, times thousands of
> installations (openQA tests
Chris,
Thanks for your feedback and comments, it's very valuable to me.
In my previous message, I mentioned that CPU is *underutilized* during
installation. I haven't investigated further why, but I suspect it's due to
the inefficiency caused by the usage of the *loop* device and/or
inefficiency
On Sun, Jan 19, 2020 at 8:41 AM Bohdan Khomutskyi wrote:
>
> Hello,
>
> Thanks everyone for posting feedback.
> More benchmarking results are available at
> https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS, including
> the 'plain' SquashFS filesystem.
> After performing the
Hello,
Thanks everyone for posting feedback.
More benchmarking results are available at
https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS, including
the 'plain' SquashFS filesystem.
After performing the tests, I personally recommend to use xz compression
with 1MiB block size,
On Sun, Jan 12, 2020 at 5:46 PM Bohdan Khomutskyi
wrote:
> Hello,
>
> I posted more benchmark results in this article:
> https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
>
> In short, bigger block size and higher compression ratio does not increase
> the installation time for
On Sun, Jan 12, 2020 at 3:53 PM Samuel Sieb wrote:
>
> On 1/7/20 11:16 AM, Kevin Kofler wrote:
> > Chris Murphy wrote:
> >> Stacked images on the same media functionality is in the kernel, it's
> >> not complicated, it's well tested, doesn't require any gymnastics in
> >> the initramfs - your
On 1/7/20 11:16 AM, Kevin Kofler wrote:
Chris Murphy wrote:
Stacked images on the same media functionality is in the kernel, it's
not complicated, it's well tested, doesn't require any gymnastics in
the initramfs - your bootloader entries can each point to different
root=UUIDs and image
On Sun, Jan 12, 2020 at 9:45 AM Bohdan Khomutskyi
wrote:
> Hello,
>
> I posted more benchmark results in this article:
> https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
>
Cool!
Do you have any tests to compare plain squashfs xz with zstd? The nested
ext4 stuff is really
On Sun, Jan 12, 2020 at 05:44:33PM +0100, Bohdan Khomutskyi wrote:
> Hello,
>
> I posted more benchmark results in this article:
> https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
>
> In short, bigger block size and higher compression ratio does not increase
> the installation
Hello,
I posted more benchmark results in this article:
https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS
In short, bigger block size and higher compression ratio does not increase
the installation time for Fedora Workstation. I saw the opposite effect.
The Zstd compression
Thanks everyone for your comments. These are all valid concerns.
I filed a new change proposal at
https://fedoraproject.org/wiki/Category:Changes/OptimizeSquashFS.
I'll run more benchmarks, including using Zstd compression algorithm, and
will post results.
Hopefully this weekend, I'll also try
>
> Why stacked images? Consider a single base.img that's maybe 1G, and
> now you don't have to do separate composes for server, cloud, GNOME,
> KDE, Cinnamon, LXQt, Astronomy that repeat a lot of the same steps,
> including expensive steps like compressing the same things over and
> over again.
On Tue, 7 Jan 2020 at 18:07, Martin Kolman wrote:
> IIRC the speedups in compression and decompression
> speed we got for RPMs[0] with zstd were pretty nice
If it helps the argument, at the moment 99.7% of the time building the
AppStream metadata is spent decompressing the RPMs. If zstd helps
On Sun, Jan 5, 2020, at 12:08 PM, Chris Murphy wrote:
> I've pretty much concluded Fedora is best off dropping the nested ext4
> in favor of plain squashfs, and using zstd.
Fedora CoreOS already uses zstd for squashfs:
On Tue, Jan 7, 2020 at 4:21 PM Kevin Kofler wrote:
> Kamil Paral wrote:
> > Well for the general user, everything is one-time. One download, one
> write
> > to USB, one install. Saving a minute in one step and adding it to a
> > different step doesn't really matter, it's the same sum overall
Chris Murphy wrote:
> Even at 8% bigger it would be worth it. And probably 16%.
I disagree. We need to stop treating bloat like a feature.
And please see my other replies for why this is a particularly bad tradeoff
in this particular case.
> Gaining additional features, like on the fly
Gary Buhrmaster wrote:
> While not exactly the same, the measured increase in size
> by the Arch community for their packaging by moving from
> xz to zstd was ~0.8% (and gaining a huge reduction in CPU
> utilization at the decompress end).
I don't know what xz settings Arch was using, but in the
Chris Murphy wrote:
> It's untenable to consider ISO size alone. It is a legitimate concern, but
> it can't be reasonable to soak every single CPU, times thousands. You're
> willing to exchange less download time for longer install time and higher
> energy demand, but there are quite a lot of
Brian C. Lane wrote:
> Yes, according to the manpage it supports xattrs.
Does that include file capabilities and ACLs?
Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to
On Tue, Jan 07, 2020 at 09:56:21AM +0100, Kevin Kofler wrote:
> Brian C. Lane wrote:
> > I agree with Chris here, I think we should make the switch to plain
> > squashfs unless someone can come up something dramatic that it will
> > break :)
>
> Does SquashFS support all the advanced features
On Tue, Jan 7, 2020 at 9:58 AM Gary Buhrmaster
wrote:
>
> On Tue, Jan 7, 2020 at 9:00 AM Kevin Kofler wrote:
>
> > I think increasing the size of the live images, also affecting the download
> > time and the time to write the image to media (even USB sticks are not
> > instant), to get a
On Tue, 2020-01-07 at 11:20 -0700, Chris Murphy wrote:
> On Tue, Jan 7, 2020 at 11:07 AM Martin Kolman wrote:
> > On Mon, 2020-01-06 at 16:35 -0800, Brian C. Lane wrote:
> > > On Sun, Jan 05, 2020 at 10:08:07AM -0700, Chris Murphy wrote:
> > > > I've pretty much concluded Fedora is best off
On Tue, Jan 7, 2020 at 11:07 AM Martin Kolman wrote:
>
> On Mon, 2020-01-06 at 16:35 -0800, Brian C. Lane wrote:
> > On Sun, Jan 05, 2020 at 10:08:07AM -0700, Chris Murphy wrote:
> > > I've pretty much concluded Fedora is best off dropping the nested ext4
> > > in favor of plain squashfs, and
On Mon, 2020-01-06 at 16:35 -0800, Brian C. Lane wrote:
> On Sun, Jan 05, 2020 at 10:08:07AM -0700, Chris Murphy wrote:
> > I've pretty much concluded Fedora is best off dropping the nested ext4
> > in favor of plain squashfs, and using zstd. It's not required to do
> > both, but the benefit is
On Tue, Jan 7, 2020 at 9:00 AM Kevin Kofler wrote:
> I think increasing the size of the live images, also affecting the download
> time and the time to write the image to media (even USB sticks are not
> instant), to get a one-time installation speedup is a very bad tradeoff.
While not exactly
It's untenable to consider ISO size alone. It is a legitimate concern, but it
can't be reasonable to soak every single CPU, times thousands. You're willing
to exchange less download time for longer install time and higher energy
demand, but there are quite a lot of other uses occurring that
Kamil Paral wrote:
> Well for the general user, everything is one-time. One download, one write
> to USB, one install. Saving a minute in one step and adding it to a
> different step doesn't really matter, it's the same sum overall (unless
> you pay considerable money for the extra downloaded
On Tue, Jan 7, 2020 at 10:01 AM Kevin Kofler wrote:
> Chris Murphy wrote:
> > Has zstandard been evaluated? In my testing of images compressed with
> > zstd, the CPU hit is cut by more than 50%, and is no longer a
> > bottleneck during installations. Image size does increase, although I
> >
Chris Murphy wrote:
> Has zstandard been evaluated? In my testing of images compressed with
> zstd, the CPU hit is cut by more than 50%, and is no longer a
> bottleneck during installations. Image size does increase, although I
> haven't tested mksquashfs block size higher than 256K.
I think
Brian C. Lane wrote:
> I agree with Chris here, I think we should make the switch to plain
> squashfs unless someone can come up something dramatic that it will
> break :)
Does SquashFS support all the advanced features that are needed, such as
extended attributes (used at least by SELinux),
On Sun, Jan 05, 2020 at 10:08:07AM -0700, Chris Murphy wrote:
> I've pretty much concluded Fedora is best off dropping the nested ext4
> in favor of plain squashfs, and using zstd. It's not required to do
> both, but the benefit is additive and significant. The work in dracut
> and lorax to
On Sun, Jan 5, 2020 at 1:24 PM Bohdan Khomutskyi
wrote:
> Summary
>
> Improve compression ratio of SquashFS filesystem on the installation media.
>
Hi Bohdan, as a member of QA, I'll happily support any proposal that
improves the installation speed (the image size is not that important from
my
On Sun, Jan 5, 2020 at 7:24 AM Bohdan Khomutskyi wrote:
>
> I was unable to create an article in Fedora wiki system.
>
Since you don't have any non-CLA groups in FAS, I have added you to
the wikiedit group. Please add this to the wiki ASAP. This proposal is
past the deadline for Fedora 32
Hi,
it does not look you follow the formal "Changes policy". The process for
submitting change proposals is described here:
https://docs.fedoraproject.org/en-US/program_management/changes_policy/
Vít
Dne 05. 01. 20 v 13:23 Bohdan Khomutskyi napsal(a):
>
>
> Summary
>
> Improve compression
On Sunday, January 5, 2020 5:23:12 AM MST Bohdan Khomutskyi wrote:
> Summary
>
> Improve compression ratio of SquashFS filesystem on the installation media.
> Owner
>
> Name: Bohdan Khomutskyi
>
> Email: bkhom...@redhat.com
> Current status
>
> Targeted release: I propose this change for
On Sun, Jan 5, 2020 at 5:24 AM Bohdan Khomutskyi wrote:
>
> Summary
>
> Improve compression ratio of SquashFS filesystem on the installation media.
On the issues of Fedora ISOs using excessive CPU, related to lzma decompression:
https://bugzilla.redhat.com/show_bug.cgi?id=1717728
50 matches
Mail list logo
