Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On 07/20/22 11:29, Richard W.M. Jones wrote: > On Wed, Jul 20, 2022 at 11:23:14AM +0200, Laszlo Ersek wrote: >> I think we could modify libguestfs to test "setfiles" for the "-T" >> option -- with setfiles_has_option() --, and if it's present, pass "-T >> 0". However, for that to be actually useful, we'd need to start up the >> appliance with multiple VCPUs -- using guestfs_set_smp(). > > For virt-v2v we actually do this already: > > https://github.com/libguestfs/virt-v2v/blob/2fbd578b4e6884a23063ad67ee36f02c4eb6c668/convert/convert.ml#L58 > > It's pretty much required for dracut to finish in a reasonable time. > >> ... Apparently, virt-customize already exposes an "--smp" option for >> that, so we might get away with modifying libguestfs only. > > Right, for the other tools users would need to add --smp to get the > benefits, but AIUI there's no drawback to using -T 0 even if running > on a single core? I think that's correct, yes. It should cause selinux_restorecon_parallel() to call selinux_restorecon_common() with the thread count set to 1 from the "nthreads" parameter, rather than selinux_restorecon() calling the same common function with the thread count set from the open-coded constant 1. Laszlo ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Wed, Jul 20, 2022 at 11:23:14AM +0200, Laszlo Ersek wrote: > I think we could modify libguestfs to test "setfiles" for the "-T" > option -- with setfiles_has_option() --, and if it's present, pass "-T > 0". However, for that to be actually useful, we'd need to start up the > appliance with multiple VCPUs -- using guestfs_set_smp(). For virt-v2v we actually do this already: https://github.com/libguestfs/virt-v2v/blob/2fbd578b4e6884a23063ad67ee36f02c4eb6c668/convert/convert.ml#L58 It's pretty much required for dracut to finish in a reasonable time. > ... Apparently, virt-customize already exposes an "--smp" option for > that, so we might get away with modifying libguestfs only. Right, for the other tools users would need to add --smp to get the benefits, but AIUI there's no drawback to using -T 0 even if running on a single core? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On 07/19/22 21:21, Richard W.M. Jones wrote:
> On Fri, Jul 15, 2022 at 05:42:35PM -0400, Ben Cotton wrote:
>> https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel
>>
>> This document represents a proposed Change. As part of the Changes
>> process, proposals are publicly announced in order to receive
>> community feedback. This proposal will only be implemented if approved
>> by the Fedora Engineering Steering Committee.
>>
>>
>> == Summary ==
>> After a system's SELinux mode is switched from disabled to enabled, or
>> after an administrator runs `fixfiles onboot`, SELinux autorelabel
>> will be run in parallel by default.
>>
>> == Owner ==
>> * Name: [[User:plautrba| Petr Lautrbach]]
>> * Email: plaut...@redhat.com
>>
>>
>> == Detailed Description ==
>> SELinux tools `restorecon` and `fixfiles` recently gained the ability
>> to relabel files in parallel using the `-T nthreads` option. This
>> option is currently not used in the automatic relabel after reboot.
>> When users want/need the parallel relabeling they have to specify the
>> option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T
>> 0` (0 == use all available CPU cores) will be the default for
>> `fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to
>> force it to use only one thread.
>>
>> The rationale is that when autorelabel runs, there are no other
>> resource-intensive processes running on the system, so it's fine (and
>> actually better) to use all available parallelism to speed up the task
>> and get to a fully booted system faster.
>>
>>
>> == Benefit to Fedora ==
>> Faster reboot after switching back to an SELinux enabled system or
>> when triggering autorelabel explicitly.
> [...]
>> == Upgrade/compatibility impact ==
>>
>>
>> == How To Test ==
>> # boot with SELinux disabled - add `selinux=0` to the kernel command line
>> # reboot
>> # store the time it took
>> # run `fixfiles -T 1 onboot`
>> # reboot
>> # the latter reboot should take longer time
> [...]
>
> I wonder if we can use this in virt tools & virt-v2v:
>
> https://github.com/libguestfs/libguestfs/blob/master/daemon/selinux-relabel.c
>
> We actually use setfiles instead of fixfiles. setfiles appears to
> have no -T option unfortunately. Is there a reason why setfiles
> doesn't have / need this option?
Upstream setfiles(8) does support the parallelized operation.
When I was working on the patches for RHBZ#1794518 (-> selinux commit
range d108226d3fc6..2b6f7e969cf6), the selinux library exposed two APIs:
- selinux_restorecon(3)
- selinux_restorecon_parallel(3)
Internally, both call selinux_restorecon_common(), just with a different
thread count (open-coded "1", versus forwarding "nthreads", respectively).
The setfiles(8) utility had already called the newer API since commit
93902fc8340f ("setfiles/restorecon: support parallel relabeling",
2021-11-23). This commit was going to be released in selinux-3.4; Fedora
35 currently ships libselinux-3.3-1.fc35.x86_64. The latest build in
Koji for Fedora 36 is only libselinux-3.3-4.fc36, too.
I think we could modify libguestfs to test "setfiles" for the "-T"
option -- with setfiles_has_option() --, and if it's present, pass "-T
0". However, for that to be actually useful, we'd need to start up the
appliance with multiple VCPUs -- using guestfs_set_smp().
... Apparently, virt-customize already exposes an "--smp" option for
that, so we might get away with modifying libguestfs only.
Thanks
Laszlo
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Tue, Jul 19, 2022 at 10:11:05PM +0200, Ondrej Mosnacek wrote: > On Tue, Jul 19, 2022 at 9:21 PM Richard W.M. Jones wrote: > > On Fri, Jul 15, 2022 at 05:42:35PM -0400, Ben Cotton wrote: > > > https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel > > > > > > This document represents a proposed Change. As part of the Changes > > > process, proposals are publicly announced in order to receive > > > community feedback. This proposal will only be implemented if approved > > > by the Fedora Engineering Steering Committee. > > > > > > > > > == Summary == > > > After a system's SELinux mode is switched from disabled to enabled, or > > > after an administrator runs `fixfiles onboot`, SELinux autorelabel > > > will be run in parallel by default. > > > > > > == Owner == > > > * Name: [[User:plautrba| Petr Lautrbach]] > > > * Email: plaut...@redhat.com > > > > > > > > > == Detailed Description == > > > SELinux tools `restorecon` and `fixfiles` recently gained the ability > > > to relabel files in parallel using the `-T nthreads` option. This > > > option is currently not used in the automatic relabel after reboot. > > > When users want/need the parallel relabeling they have to specify the > > > option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T > > > 0` (0 == use all available CPU cores) will be the default for > > > `fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to > > > force it to use only one thread. > > > > > > The rationale is that when autorelabel runs, there are no other > > > resource-intensive processes running on the system, so it's fine (and > > > actually better) to use all available parallelism to speed up the task > > > and get to a fully booted system faster. > > > > > > > > > == Benefit to Fedora == > > > Faster reboot after switching back to an SELinux enabled system or > > > when triggering autorelabel explicitly. > > [...] > > > == Upgrade/compatibility impact == > > > > > > > > > == How To Test == > > > # boot with SELinux disabled - add `selinux=0` to the kernel command line > > > # reboot > > > # store the time it took > > > # run `fixfiles -T 1 onboot` > > > # reboot > > > # the latter reboot should take longer time > > [...] > > > > I wonder if we can use this in virt tools & virt-v2v: > > > > https://github.com/libguestfs/libguestfs/blob/master/daemon/selinux-relabel.c > > > > We actually use setfiles instead of fixfiles. setfiles appears to > > have no -T option unfortunately. Is there a reason why setfiles > > doesn't have / need this option? > > Both setfiles and restorecon also have the -T option, as long as you > are running a recent enough Fedora (36+, AFAIK). Yes, indeed it does - I missed it in the man page. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com nbdkit - Flexible, fast NBD server with plugins https://gitlab.com/nbdkit/nbdkit ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Tue, Jul 19, 2022 at 9:21 PM Richard W.M. Jones wrote: > On Fri, Jul 15, 2022 at 05:42:35PM -0400, Ben Cotton wrote: > > https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel > > > > This document represents a proposed Change. As part of the Changes > > process, proposals are publicly announced in order to receive > > community feedback. This proposal will only be implemented if approved > > by the Fedora Engineering Steering Committee. > > > > > > == Summary == > > After a system's SELinux mode is switched from disabled to enabled, or > > after an administrator runs `fixfiles onboot`, SELinux autorelabel > > will be run in parallel by default. > > > > == Owner == > > * Name: [[User:plautrba| Petr Lautrbach]] > > * Email: plaut...@redhat.com > > > > > > == Detailed Description == > > SELinux tools `restorecon` and `fixfiles` recently gained the ability > > to relabel files in parallel using the `-T nthreads` option. This > > option is currently not used in the automatic relabel after reboot. > > When users want/need the parallel relabeling they have to specify the > > option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T > > 0` (0 == use all available CPU cores) will be the default for > > `fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to > > force it to use only one thread. > > > > The rationale is that when autorelabel runs, there are no other > > resource-intensive processes running on the system, so it's fine (and > > actually better) to use all available parallelism to speed up the task > > and get to a fully booted system faster. > > > > > > == Benefit to Fedora == > > Faster reboot after switching back to an SELinux enabled system or > > when triggering autorelabel explicitly. > [...] > > == Upgrade/compatibility impact == > > > > > > == How To Test == > > # boot with SELinux disabled - add `selinux=0` to the kernel command line > > # reboot > > # store the time it took > > # run `fixfiles -T 1 onboot` > > # reboot > > # the latter reboot should take longer time > [...] > > I wonder if we can use this in virt tools & virt-v2v: > > https://github.com/libguestfs/libguestfs/blob/master/daemon/selinux-relabel.c > > We actually use setfiles instead of fixfiles. setfiles appears to > have no -T option unfortunately. Is there a reason why setfiles > doesn't have / need this option? Both setfiles and restorecon also have the -T option, as long as you are running a recent enough Fedora (36+, AFAIK). -- Ondrej Mosnacek Senior Software Engineer, Linux Security - SELinux kernel Red Hat, Inc. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Fri, Jul 15, 2022 at 05:42:35PM -0400, Ben Cotton wrote: > https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel > > This document represents a proposed Change. As part of the Changes > process, proposals are publicly announced in order to receive > community feedback. This proposal will only be implemented if approved > by the Fedora Engineering Steering Committee. > > > == Summary == > After a system's SELinux mode is switched from disabled to enabled, or > after an administrator runs `fixfiles onboot`, SELinux autorelabel > will be run in parallel by default. > > == Owner == > * Name: [[User:plautrba| Petr Lautrbach]] > * Email: plaut...@redhat.com > > > == Detailed Description == > SELinux tools `restorecon` and `fixfiles` recently gained the ability > to relabel files in parallel using the `-T nthreads` option. This > option is currently not used in the automatic relabel after reboot. > When users want/need the parallel relabeling they have to specify the > option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T > 0` (0 == use all available CPU cores) will be the default for > `fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to > force it to use only one thread. > > The rationale is that when autorelabel runs, there are no other > resource-intensive processes running on the system, so it's fine (and > actually better) to use all available parallelism to speed up the task > and get to a fully booted system faster. > > > == Benefit to Fedora == > Faster reboot after switching back to an SELinux enabled system or > when triggering autorelabel explicitly. [...] > == Upgrade/compatibility impact == > > > == How To Test == > # boot with SELinux disabled - add `selinux=0` to the kernel command line > # reboot > # store the time it took > # run `fixfiles -T 1 onboot` > # reboot > # the latter reboot should take longer time [...] I wonder if we can use this in virt tools & virt-v2v: https://github.com/libguestfs/libguestfs/blob/master/daemon/selinux-relabel.c We actually use setfiles instead of fixfiles. setfiles appears to have no -T option unfortunately. Is there a reason why setfiles doesn't have / need this option? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
Gary Buhrmaster writes:
> On Mon, Jul 18, 2022 at 6:44 AM Petr Lautrbach wrote:
>>
>> Dan Čermák writes:
>> >
>> > Just out of curiosity, how large is the speedup typically?
>> >
>
>>
>> It depends on the number of threads your machine has. But you could get some
>> data for comparison using `fixfiles -T 1 restore` and `fixfiles -T 0
>> restore` on a running system. The following times are reported on my
>> workstation:
>>
>
> Has anyone run such a test on a system using
> classic ("spinning rust") HDDs? It is sometimes
> the case that parallelizing activities that are I/O
> intensive can result in excessive seek activity
> that can result in rather elongated elapsed times
> (much worse than single threaded operation).
We have only one machine with a rotary HDD and 4 threads:
# for T in 1 0 4; do chcon -R -t root_t /etc/bad-label/; sync; for i in $(seq 1
3); do echo $i > /proc/sys/vm/drop_caches; done; echo -T $T; time fixfiles -F
-T $T restore; done
-T 1
real3m32,046s
user2m0,068s
sys 0m24,005s
-T 0
real2m32,859s
user2m9,265s
sys 0m26,775s
-T 4
real2m38,355s
user2m9,695s
sys 0m27,042s
Petr
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
Unfortunately, the only machine I have access to with an HDD as the
system drive is running Ubuntu 20.04 LTS, which doesn’t have a
multi-threaded version of fixfiles. So I can’t test this on an HDD-only
system.
Still, here are the timings on a 4-core Atom machine with a SATA SSD and
a large SATA HDD data drive mounted at /srv:
$ sudo bash -c 'sync; for i in $(seq 1 3); do echo $i >
/proc/sys/vm/drop_caches; done; time fixfiles -T 1 restore'
[…]
real 6m39.521s
user 5m6.599s
sys 0m42.000s
$ sudo bash -c 'sync; for i in $(seq 1 3); do echo $i >
/proc/sys/vm/drop_caches; done; time fixfiles -T 0 restore'
[…]
real 3m16.602s
user 5m25.444s
sys 0m50.843s
I took care to drop various kernel caches, especially including dentry
and inode caches, to make this more reproducible.
Here are the timings on a very modern 16-core machine with NVMe SSD storage:
$ sudo bash -c 'sync; for i in $(seq 1 3); do echo $i >
/proc/sys/vm/drop_caches; done; time fixfiles -T 1 restore'
[…]
real 3m17.277s
user 2m42.447s
sys 0m24.403s
$ sudo bash -c 'sync; for i in $(seq 1 3); do echo $i >
/proc/sys/vm/drop_caches; done; time fixfiles -T 0 restore'
[…]
real 0m58.154s
user 4m27.209s
sys 0m58.127s
On 7/18/22 11:48, Gary Buhrmaster wrote:
On Mon, Jul 18, 2022 at 6:44 AM Petr Lautrbach wrote:
Dan Čermák writes:
Just out of curiosity, how large is the speedup typically?
It depends on the number of threads your machine has. But you could get some
data for comparison using `fixfiles -T 1 restore` and `fixfiles -T 0
restore` on a running system. The following times are reported on my
workstation:
Has anyone run such a test on a system using
classic ("spinning rust") HDDs? It is sometimes
the case that parallelizing activities that are I/O
intensive can result in excessive seek activity
that can result in rather elongated elapsed times
(much worse than single threaded operation).
___
devel mailing list --devel@lists.fedoraproject.org
To unsubscribe send an email todevel-le...@lists.fedoraproject.org
Fedora Code of
Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines
List
Archives:https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report
it:https://pagure.io/fedora-infrastructure___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Mon, Jul 18, 2022 at 11:50 AM Gary Buhrmaster
wrote:
>
> On Mon, Jul 18, 2022 at 6:44 AM Petr Lautrbach wrote:
> >
> > Dan Čermák writes:
> > >
> > > Just out of curiosity, how large is the speedup typically?
> > >
>
> >
> > It depends on the number of threads your machine has. But you could get some
> > data for comparison using `fixfiles -T 1 restore` and `fixfiles -T 0
> > restore` on a running system. The following times are reported on my
> > workstation:
> >
>
> Has anyone run such a test on a system using
> classic ("spinning rust") HDDs? It is sometimes
> the case that parallelizing activities that are I/O
> intensive can result in excessive seek activity
> that can result in rather elongated elapsed times
> (much worse than single threaded operation).
If that turns out to be the case, it should be possible to detect the
use of an SSD vs. a spinning-disc HDD and trigger the `-T` argument
appropriately. I don't know if the Change Proposers have plans for
that.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Mon, Jul 18, 2022 at 6:44 AM Petr Lautrbach wrote:
>
> Dan Čermák writes:
> >
> > Just out of curiosity, how large is the speedup typically?
> >
>
> It depends on the number of threads your machine has. But you could get some
> data for comparison using `fixfiles -T 1 restore` and `fixfiles -T 0
> restore` on a running system. The following times are reported on my
> workstation:
>
Has anyone run such a test on a system using
classic ("spinning rust") HDDs? It is sometimes
the case that parallelizing activities that are I/O
intensive can result in excessive seek activity
that can result in rather elongated elapsed times
(much worse than single threaded operation).
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On Mon, Jul 18, 2022 at 8:44 AM Petr Lautrbach wrote: > Dan Čermák writes: > > > On July 15, 2022 9:42:35 PM UTC, Ben Cotton wrote: > >>https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel > >> > >>This document represents a proposed Change. As part of the Changes > >>process, proposals are publicly announced in order to receive > >>community feedback. This proposal will only be implemented if approved > >>by the Fedora Engineering Steering Committee. > >> > >> > >>== Summary == > >>After a system's SELinux mode is switched from disabled to enabled, or > >>after an administrator runs `fixfiles onboot`, SELinux autorelabel > >>will be run in parallel by default. > >> > >>== Owner == > >>* Name: [[User:plautrba| Petr Lautrbach]] > >>* Email: plaut...@redhat.com > >> > >> > >>== Detailed Description == > >>SELinux tools `restorecon` and `fixfiles` recently gained the ability > >>to relabel files in parallel using the `-T nthreads` option. This > >>option is currently not used in the automatic relabel after reboot. > >>When users want/need the parallel relabeling they have to specify the > >>option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T > >>0` (0 == use all available CPU cores) will be the default for > >>`fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to > >>force it to use only one thread. > >> > >>The rationale is that when autorelabel runs, there are no other > >>resource-intensive processes running on the system, so it's fine (and > >>actually better) to use all available parallelism to speed up the task > >>and get to a fully booted system faster. > >> > >> > >>== Benefit to Fedora == > >>Faster reboot after switching back to an SELinux enabled system or > >>when triggering autorelabel explicitly. > > > > Just out of curiosity, how large is the speedup typically? > > > >> > > > It depends on the number of threads your machine has. But you could get some > data for comparison using `fixfiles -T 1 restore` and `fixfiles -T 0 > restore` on a running system. The following times are reported on my > workstation: > > [root@P1 ~]# time fixfiles -T 0 restore > Relabeling / /boot /dev /dev/hugepages /dev/mqueue /dev/pts /dev/shm /home > /run /run/user/1000 /sys /sys/fs/cgroup /sys/fs/pstore /sys/kernel/debug > /sys/kernel/debug/tracing /sys/kernel/tracing /tmp /var > / 100.0% > ... > real1m8.488s > user9m24.755s > sys 0m25.424s > > [root@P1 ~]# time fixfiles -T 1 restore > Relabeling / /boot /dev /dev/hugepages /dev/mqueue /dev/pts /dev/shm /home > /run /run/user/1000 /sys /sys/fs/cgroup /sys/fs/pstore /sys/kernel/debug > /sys/kernel/debug/tracing /sys/kernel/tracing /tmp /var > / 100.0% > ... > real4m5.450s > user3m55.017s > sys 0m10.088s Also see the original commit message, which compares the run time for different thread counts on a 32-core machine: https://github.com/SELinuxProject/selinux/commit/93902fc8340f8a6ee5ba69ccb150d47918aad226 It doesn't scale perfectly with the number of cores, but it can speed up the relabeling up to ~18 times if you have a beefy machine. I updated the "Benefit to Fedora" section of the proposal with this info. -- Ondrej Mosnacek Senior Software Engineer, Linux Security - SELinux kernel Red Hat, Inc. ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
Dan Čermák writes: > On July 15, 2022 9:42:35 PM UTC, Ben Cotton wrote: >>https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel >> >>This document represents a proposed Change. As part of the Changes >>process, proposals are publicly announced in order to receive >>community feedback. This proposal will only be implemented if approved >>by the Fedora Engineering Steering Committee. >> >> >>== Summary == >>After a system's SELinux mode is switched from disabled to enabled, or >>after an administrator runs `fixfiles onboot`, SELinux autorelabel >>will be run in parallel by default. >> >>== Owner == >>* Name: [[User:plautrba| Petr Lautrbach]] >>* Email: plaut...@redhat.com >> >> >>== Detailed Description == >>SELinux tools `restorecon` and `fixfiles` recently gained the ability >>to relabel files in parallel using the `-T nthreads` option. This >>option is currently not used in the automatic relabel after reboot. >>When users want/need the parallel relabeling they have to specify the >>option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T >>0` (0 == use all available CPU cores) will be the default for >>`fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to >>force it to use only one thread. >> >>The rationale is that when autorelabel runs, there are no other >>resource-intensive processes running on the system, so it's fine (and >>actually better) to use all available parallelism to speed up the task >>and get to a fully booted system faster. >> >> >>== Benefit to Fedora == >>Faster reboot after switching back to an SELinux enabled system or >>when triggering autorelabel explicitly. > > Just out of curiosity, how large is the speedup typically? > >> It depends on the number of threads your machine has. But you could get some data for comparison using `fixfiles -T 1 restore` and `fixfiles -T 0 restore` on a running system. The following times are reported on my workstation: [root@P1 ~]# time fixfiles -T 0 restore Relabeling / /boot /dev /dev/hugepages /dev/mqueue /dev/pts /dev/shm /home /run /run/user/1000 /sys /sys/fs/cgroup /sys/fs/pstore /sys/kernel/debug /sys/kernel/debug/tracing /sys/kernel/tracing /tmp /var / 100.0% ... real1m8.488s user9m24.755s sys 0m25.424s [root@P1 ~]# time fixfiles -T 1 restore Relabeling / /boot /dev /dev/hugepages /dev/mqueue /dev/pts /dev/shm /home /run /run/user/1000 /sys /sys/fs/cgroup /sys/fs/pstore /sys/kernel/debug /sys/kernel/debug/tracing /sys/kernel/tracing /tmp /var / 100.0% ... real4m5.450s user3m55.017s sys 0m10.088s Petr >>== Scope == >>* Proposal owners: >>** Update `/usr/libexec/selinux/selinux-autorelabel` to use `-T 0` by default. >> >>* Other developers: >>* Release engineering: >>* Policies and guidelines: N/A (not needed for this Change) >>* Trademark approval: N/A (not needed for this Change) >>* Alignment with Objectives: >> >> >>== Upgrade/compatibility impact == >> >> >>== How To Test == >># boot with SELinux disabled - add `selinux=0` to the kernel command line >># reboot >># store the time it took >># run `fixfiles -T 1 onboot` >># reboot >># the latter reboot should take longer time >> >> >>== User Experience == >>Systems should be up and running faster after SELinux autorelabel. >> >>== Dependencies == >> >> >>== Contingency Plan == >>* Contingency mechanism: (What to do? Who will do it?) N/A (not a >>System Wide Change) >>* Contingency deadline: N/A (not a System Wide Change) >>* Blocks release? N/A (not a System Wide Change), Yes/No >> >>== Documentation == >> >>N/A (not a System Wide Change) >> >> > ___ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Re: F37 proposal: SELinux Parallel Autorelabel (Self-Contained Change proposal)
On July 15, 2022 9:42:35 PM UTC, Ben Cotton wrote: >https://fedoraproject.org/wiki/Changes/SELinux_Parallel_Autorelabel > >This document represents a proposed Change. As part of the Changes >process, proposals are publicly announced in order to receive >community feedback. This proposal will only be implemented if approved >by the Fedora Engineering Steering Committee. > > >== Summary == >After a system's SELinux mode is switched from disabled to enabled, or >after an administrator runs `fixfiles onboot`, SELinux autorelabel >will be run in parallel by default. > >== Owner == >* Name: [[User:plautrba| Petr Lautrbach]] >* Email: plaut...@redhat.com > > >== Detailed Description == >SELinux tools `restorecon` and `fixfiles` recently gained the ability >to relabel files in parallel using the `-T nthreads` option. This >option is currently not used in the automatic relabel after reboot. >When users want/need the parallel relabeling they have to specify the >option explicitly (e.g. `fixfiles -T 0 onboot`). With this change `-T >0` (0 == use all available CPU cores) will be the default for >`fixfiles onboot` and users will have to use `fixfiles -T 1 onboot` to >force it to use only one thread. > >The rationale is that when autorelabel runs, there are no other >resource-intensive processes running on the system, so it's fine (and >actually better) to use all available parallelism to speed up the task >and get to a fully booted system faster. > > >== Benefit to Fedora == >Faster reboot after switching back to an SELinux enabled system or >when triggering autorelabel explicitly. Just out of curiosity, how large is the speedup typically? > >== Scope == >* Proposal owners: >** Update `/usr/libexec/selinux/selinux-autorelabel` to use `-T 0` by default. > >* Other developers: >* Release engineering: >* Policies and guidelines: N/A (not needed for this Change) >* Trademark approval: N/A (not needed for this Change) >* Alignment with Objectives: > > >== Upgrade/compatibility impact == > > >== How To Test == ># boot with SELinux disabled - add `selinux=0` to the kernel command line ># reboot ># store the time it took ># run `fixfiles -T 1 onboot` ># reboot ># the latter reboot should take longer time > > >== User Experience == >Systems should be up and running faster after SELinux autorelabel. > >== Dependencies == > > >== Contingency Plan == >* Contingency mechanism: (What to do? Who will do it?) N/A (not a >System Wide Change) >* Contingency deadline: N/A (not a System Wide Change) >* Blocks release? N/A (not a System Wide Change), Yes/No > >== Documentation == > >N/A (not a System Wide Change) > > ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
