Re: Proposal: Disable autoload of compat_xyz modules

[Paul Goyette]

On Wed, 2 Aug 2017, Michael van Elst wrote:


p...@whooppee.com (Paul Goyette) writes:


At least for x86 architectures, you can easily edit /boot.cfg to include
the required load commands.  This causes the boot loader to "push" the
modules before init gets started.


You can also load modules listed in /etc/modules.conf at boot time.


Yes, /etc/modules.conf will also work, however modules listed there are 
loaded at a much later point in time.



But unlike autoloading the machine has to reboot if you want to
change the loaded modules.


If you're using /etc/modules.conf, why not just modunload the modules 
you want to remove?  (Modules "pushed" from /boot.conf cannot be 
unloaded, as far as I understand.)



+--+--++
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:  |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee dot com   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd dot org |
+--+--++

Re: Proposal: Disable autoload of compat_xyz modules

[Michael van Elst]

p...@whooppee.com (Paul Goyette) writes:

>At least for x86 architectures, you can easily edit /boot.cfg to include 
>the required load commands.  This causes the boot loader to "push" the 
>modules before init gets started.

You can also load modules listed in /etc/modules.conf at boot time.
But unlike autoloading the machine has to reboot if you want to
change the loaded modules.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: Proposal: Disable autoload of compat_xyz modules

[Paul Goyette]

On Wed, 2 Aug 2017, Michael van Elst wrote:


e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes:


Do you, or does anyone you know, rely on any of these modules?

While I do use both linux and linux32 (to execute the proprietary TSM client),
I could just modload them if I wouldn't build my own kernel anyway.


The difference is to have a fix configuration to load it while booting
or to load and unload it on demand.


At least for x86 architectures, you can easily edit /boot.cfg to include 
the required load commands.  This causes the boot loader to "push" the 
modules before init gets started.


I do this regularly with ffs & wapbl (these modules are NOT included in 
my custom kernel configs)!


# caqt /boot.cfg
load=wapbl
load=ffs
menu=Boot normally:rndseed /var/db/entropy-file;boot netbsd
menu=Boot single user:rndseed /var/db/entropy-file;boot netbsd -s
menu=Disable ACPI:rndseed /var/db/entropy-file;boot netbsd -2
menu=Disable ACPI and SMP:rndseed /var/db/entropy-file;boot netbsd -12
menu=Drop to boot prompt:prompt
menu=Run memory test:boot memtestplus

default=1
timeout=5
clear=1
#

+--+--++
| Paul Goyette | PGP Key fingerprint: | E-mail addresses:  |
| (Retired)| FA29 0E3B 35AF E8AE 6651 | paul at whooppee dot com   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd dot org |
+--+--++

Re: Proposal: Disable autoload of compat_xyz modules

[Michael van Elst]

e...@math.uni-bonn.de (Edgar =?iso-8859-1?B?RnXf?=) writes:

>> Do you, or does anyone you know, rely on any of these modules?
>While I do use both linux and linux32 (to execute the proprietary TSM client),
>I could just modload them if I wouldn't build my own kernel anyway.

The difference is to have a fix configuration to load it while booting
or to load and unload it on demand.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."

Re: Proposal: Disable autoload of compat_xyz modules

[Jaromír Doleček]

I like that all the arch-specific code is under sys/arch, and not randomly
spread around tree, i.e. I prefer to keep the compat things under sys/arch.

For sure, same argument could be used the opposite direction, that it would
be neater to have all the compat code together.

But IMO the arch-specific bits are more tied to the specific platform, so
it makes sense to have them together with their native counterparts - like
e.g. signal trampoline code.

Jaromir

2017-08-01 13:12 GMT+02:00 Maxime Villard :
> Yes, that's the right thing to do.
>
> I haven't rechecked, but as far as I remember, there will be some issues
with
> the compat layers that have several #ifdefs deep into the machdep code.
>
> Apart from that, several files could usefully be moved into the
appropriate
> compat_xyz directory, instead of polluting sys/arch/. For example:
>
> sys/arch/amd64/amd64/linux_*->   sys/compat/linux/arch/amd64/
> sys/arch/amd64/amd64/linux32_*  ->
sys/compat/linux32/arch/amd64/
>
> This would provide a better encapsulation.

Re: Proposal: Disable autoload of compat_xyz modules

[David Holland]

On Tue, Aug 01, 2017 at 12:44:57PM +, m...@netbsd.org wrote:
 > netbsd/mips64 runs a 64bit kernel and full 32bit userland, so
 > compat_netbsd32 is tested very heavily on it.

We should build 32-bit tests on amd64 :-)

(also we should think about how to set up to be able to compile compat
tests...)

-- 
David A. Holland
dholl...@netbsd.org

Re: Proposal: Disable autoload of compat_xyz modules

[David Holland]

On Tue, Aug 01, 2017 at 07:01:29PM +0200, Maxime Villard wrote:
 > Typically, the situation I want to avoid is this [1], where you had some
 > random compat code next to critical native procedures. Note also that the
 > recent callgate vulnerability in amd64 wouldn't have existed, had the
 > separation between native and compat code been stark.

You could just as easily argue that similar functions should be
grouped together, so an issue found in one gets fixed in all versions
instead of leading to exploits and embarrassing advisories later on.

That said, it's not wrong to have a compat/arch. In fact, after trying
out various configurations in research kernels over the years my
recommendation is to have an arch dir in each subsystem dir that needs
one (thus, sys/uvm/arch, sys/dev/arch, etc.) rather than one big one.

Of course we don't really have subsystem dirs yet either.

but big reorgs need migration to a version control system that
supports rename... (deepseagreen4)

-- 
David A. Holland
dholl...@netbsd.org

Re: Proposal: Disable autoload of compat_xyz modules

[Maxime Villard]

As you said, this argument also works the other way around: it is good to
have compat-specific code under sys/compat, and not randomly spread around
the tree.

In case you didn't notice, compat/linux/arch is already full of arch-
specific code that plays with trap frames, which by definition are tied to
the specific platform. So it's not like I'm introducing a new concept.

Given Taylor's proposal, I believe it is more important to have the compat
code together: in a modularised approach, we will certainly want all of the
linux code under compat/linux, to improve readability and maintainability.
It also has the advantage of forcing the removal of the #ifdefs in the
arch-specific code.

Typically, the situation I want to avoid is this [1], where you had some
random compat code next to critical native procedures. Note also that the
recent callgate vulnerability in amd64 wouldn't have existed, had the
separation between native and compat code been stark.

Maxime

[1] http://mail-index.netbsd.org/source-changes/2017/07/29/msg086828.html



Le 01/08/2017 à 18:34, Jaromír Doleček a écrit :

I like that all the arch-specific code is under sys/arch, and not randomly 
spread around tree, i.e. I prefer to keep the compat things under sys/arch.

For sure, same argument could be used the opposite direction, that it would be 
neater to have all the compat code together.

But IMO the arch-specific bits are more tied to the specific platform, so it 
makes sense to have them together with their native counterparts - like e.g. 
signal trampoline code.

Jaromir

2017-08-01 13:12 GMT+02:00 Maxime Villard >:
 > Yes, that's the right thing to do.
 >
 > I haven't rechecked, but as far as I remember, there will be some issues with
 > the compat layers that have several #ifdefs deep into the machdep code.
 >
 > Apart from that, several files could usefully be moved into the appropriate
 > compat_xyz directory, instead of polluting sys/arch/. For example:
 >
 > sys/arch/amd64/amd64/linux_*->   sys/compat/linux/arch/amd64/
 > sys/arch/amd64/amd64/linux32_*  ->   sys/compat/linux32/arch/amd64/
 >
 > This would provide a better encapsulation.

Re: Proposal: Disable autoload of compat_xyz modules

[Taylor R Campbell]

> Date: Tue, 1 Aug 2017 18:34:56 +0200
> From: Jaromír Doleček 
> 
> I like that all the arch-specific code is under sys/arch, and not randomly
> spread around tree, i.e. I prefer to keep the compat things under sys/arch.
> 
> For sure, same argument could be used the opposite direction, that it would
> be neater to have all the compat code together.
> 
> But IMO the arch-specific bits are more tied to the specific platform, so
> it makes sense to have them together with their native counterparts - like
> e.g. signal trampoline code.

Obviously we need a relational database of source files to maintain
physical access-path independence of the semantic relations here.

Re: Proposal: Disable autoload of compat_xyz modules

[Robert Swindells]

m...@netbsd.org wrote:
On Mon, Jul 31, 2017 at 09:39:42PM +, Taylor R Campbell wrote:
>> P.S.  The old-NetBSD, a.out, and 32-bit compat modules may be broken
>> too, and are probably not automatically tested either, but are more
>> likely to be manually tested and I'm not addressing them right now.
>> These are:
>
>netbsd/mips64 runs a 64bit kernel and full 32bit userland, so
>>compat_netbsd32 is tested very heavily on it.

Running NetBSD/sparc on 64bit hardware works like this too.

Re: ndis [was Re: Proposal: Disable autoload of compat_xyz modules]

[maya]

On Mon, Jul 31, 2017 at 06:00:18PM -0500, Jonathan A. Kollasch wrote:
> I might care about it if it worked.  I couldn't even make it work for
> wired NICs some years ago.  I have a feeling it's incomplete and/or
> broken.  I don't think it does any syscall compat stuff though.

The code and some other things claim it is compatible with NDIS 5.1, I
am under the impression that means only Windows XP or older drivers will
work.

I think manufacturers have stopped making such drivers, but they can
still be found for hardware.

Re: Proposal: Disable autoload of compat_xyz modules

[Robert Swindells]

Taylor R Campbell  wrote:
>> Date: Tue, 1 Aug 2017 00:45:24 +0200
>> From: Kamil Rytarowski 
>> 
>> I use both linux and linux32 ones.
>
>Can you argue that these and the others you listed should be
>automatically enabled by default, and that having to modload them
>would be too much of a burden?
>
>Can you say who maintains them, or volunteer to maintain them,
>including writing automatic tests with sample binaries that others can
>use to test security fixes?
>
>I want to learn more than just who uses these -- I want to estimate
>the negative impact of disabling them by default, and what effort we
>have available to make sure they continue to work reliably.

I use linux, linux32 and netbsd32 to run fairly big applications that
I think test out a good spread of features.

The Xilinx FPGA tools make use of compat_linux and compat_linux32. I
have an uncommitted patch to procfs to make the latest version work.

I regularly run CMUCL on amd64 using compat_netbsd32.

Re: Proposal: Disable autoload of compat_xyz modules

[maya]

On Mon, Jul 31, 2017 at 09:39:42PM +, Taylor R Campbell wrote:
> Can you tell me who maintains them, or if nobody does, can you
> volunteer to maintain them -- by auditing them, by testing them if
> someone else applies a security fix, by writing automatic tests with
> sample binaries that we can put into atf?

I can promise to try to solve any COMPAT_LINUX bug, although I don't
promise to succeed. I find it useful, we even have packages relying on
its existence, and its nice that they work out of the box, but adding a
note to all of them isn't bad.

Most of these packages are in pkgsrc/games, but there are some other
notable software that has been reported to run under COMPAT_LINUX
(recently manu@ added pselect6 stating it allows MATLAB R2016A to run).

> P.S.  The old-NetBSD, a.out, and 32-bit compat modules may be broken
> too, and are probably not automatically tested either, but are more
> likely to be manually tested and I'm not addressing them right now.
> These are:

netbsd/mips64 runs a 64bit kernel and full 32bit userland, so
compat_netbsd32 is tested very heavily on it.

Re: ndis [was Re: Proposal: Disable autoload of compat_xyz modules]

[Joerg Sonnenberger]

On Mon, Jul 31, 2017 at 06:00:18PM -0500, Jonathan A. Kollasch wrote:
> On Mon, Jul 31, 2017 at 10:56:41PM +, Taylor R Campbell wrote:
> > > Date: Tue, 1 Aug 2017 00:45:24 +0200
> > > From: Kamil Rytarowski 
> > > 
> > > > compat_ndis
> > > 
> > > This one perhaps could stay. It's for network driver.
> > 
> > On review, it appears this is not actually included in any GENERIC
> > kernels at all -- or even the x86 ALL kernels.  It doesn't even seem
> > to build.  It is marked as `experimental'.  Does anyone care about it?
> > 
> 
> I might care about it if it worked.  I couldn't even make it work for
> wired NICs some years ago.  I have a feeling it's incomplete and/or
> broken.  I don't think it does any syscall compat stuff though.

Correct. It is an emulation layer of the NDIS kernel API. No user
servicable parts here.

Joerg

Re: Proposal: Disable autoload of compat_xyz modules

[Edgar Fuß]

> Do you, or does anyone you know, rely on any of these modules?
While I do use both linux and linux32 (to execute the proprietary TSM client),
I could just modload them if I wouldn't build my own kernel anyway.

Re: kmem_alloc(0, f)

[Michael van Elst]

mar...@duskware.de (Martin Husemann) writes:

>An empty array still is 99% of the times a driver bug and simply caught
>when first trying the driver. All other call sides can simply avoid the
>allocation and use a NULL pointer instead.

Aren't zero sized allocations a must for Linux kernel shim compatibility?
If I remember correctly the major user was the radeon DRM code.

-- 
-- 
Michael van Elst
Internet: mlel...@serpens.de
"A potential Snark may lurk in every tree."