Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: > Hi. > > (added some people to CC) > > > On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov > wrote: > > > > Problem: > > > > When a kernel module is compiled as a separate module, some important > > information about the kernel module is available via .modinfo section of > > the module. In contrast, when the kernel module is compiled into the > > kernel, that information is not available. > > > I might be missing something, but > vmlinux provides info of builtin modules > in /sys/module/. No. There are definitely not all modules. I have a builtin sha256_generic, but I can't find him in the /sys/module. > (Looks like currently only module_param and MODULE_VERSION) > > This patch is not exactly the same, but I see a kind of overwrap. > I'd like to be sure if we want this new scheme. The /sys/module is only for running kernel. One of my use cases is to create an initrd for a new kernel. > > > Information about built-in modules is necessary in the following cases: > > > > 1. When it is necessary to find out what additional parameters can be > > passed to the kernel at boot time. > > > Actually, /sys/module//parameters/ > exposes this information. > > Doesn't it work for your purpose? No, since creating an initrd needs to know all the modalias before I get the sysfs for new kernel. Also there are no modalias at all. > > 2. When you need to know which module names and their aliases are in > > the kernel. This is very useful for creating an initrd image. > > > > Proposal: > > > > The proposed patch does not remove .modinfo section with module > > information from the vmlinux at the build time and saves it into a > > separate file after kernel linking. So, the kernel does not increase in > > size and no additional information remains in it. Information is stored > > in the same format as in the separate modules (null-terminated string > > array). Because the .modinfo section is already exported with a separate > > modules, we are not creating a new API. > > > > It can be easily read in the userspace: > > > > $ tr '\0' '\n' < kernel.builtin.modinfo > > ext4.softdep=pre: crc32c > > ext4.license=GPL > > ext4.description=Fourth Extended Filesystem > > ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, > > Theodore Ts'o and others > > ext4.alias=fs-ext4 > > ext4.alias=ext3 > > ext4.alias=fs-ext3 > > ext4.alias=ext2 > > ext4.alias=fs-ext2 > > md_mod.alias=block-major-9-* > > md_mod.alias=md > > md_mod.description=MD RAID framework > > md_mod.license=GPL > > md_mod.parmtype=create_on_open:bool > > md_mod.parmtype=start_dirty_degraded:int > > ... > > > > Co-Developed-by: Gleb Fotengauer-Malinovskiy > > Signed-off-by: Gleb Fotengauer-Malinovskiy > > Signed-off-by: Alexey Gladkov > > --- > > Makefile| 1 + > > include/linux/moduleparam.h | 12 +--- > > scripts/link-vmlinux.sh | 8 > > 3 files changed, 14 insertions(+), 7 deletions(-) > > > > diff --git a/Makefile b/Makefile > > index d5713e7b1e50..971102194c92 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -1288,6 +1288,7 @@ _modinst_: > > fi > > @cp -f $(objtree)/modules.order $(MODLIB)/ > > @cp -f $(objtree)/modules.builtin $(MODLIB)/ > > + @cp -f $(objtree)/kernel.builtin.modinfo $(MODLIB)/ > > $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst > > > > # This depmod is only for convenience to give the initial > > diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h > > index ba36506db4fb..5ba250d9172a 100644 > > --- a/include/linux/moduleparam.h > > +++ b/include/linux/moduleparam.h > > @@ -10,23 +10,21 @@ > > module name. */ > > #ifdef MODULE > > #define MODULE_PARAM_PREFIX /* empty */ > > +#define __MODULE_INFO_PREFIX /* empty */ > > #else > > #define MODULE_PARAM_PREFIX KBUILD_MODNAME "." > > +/* We cannot use MODULE_PARAM_PREFIX because some modules override it. */ > > +#define __MODULE_INFO_PREFIX KBUILD_MODNAME "." > > #endif > > > > /* Chosen so that structs with an unsigned long line up. */ > > #define MAX_PARAM_PREFIX_LEN (64 - sizeof(unsigned long)) > > > > -#ifdef MODULE > > #define __MODULE_INFO(tag, name, info) \ > > static const char __UNIQUE_ID(name)[]\ > >__used __attribute__((section(".modinfo"), unused, aligned(1)))\ > > - = __stringify(tag) "=" info > > -#else /* !MODULE */ > > -/* This struct is here for syntactic coherency, it is not used */ > > -#define __MODULE_INFO(tag, name, info) \ > > - struct __UNIQUE_ID(name) {} > > -#endif > > + = __MODULE_INFO_PREFIX __stringify(tag) "=" info > > + > > #define __MODULE_PARM_TYPE(name, _type) > > \ > >__MODULE_INFO(parmtype, name##type, #name ":" _type) > > > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
+++ Alexey Gladkov [26/03/19 18:24 +0100]: On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: Hi. (added some people to CC) (Thanks Masahiro for the CC!) On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov wrote: > > Problem: > > When a kernel module is compiled as a separate module, some important > information about the kernel module is available via .modinfo section of > the module. In contrast, when the kernel module is compiled into the > kernel, that information is not available. I might be missing something, but vmlinux provides info of builtin modules in /sys/module/. No. There are definitely not all modules. I have a builtin sha256_generic, but I can't find him in the /sys/module. Yeah, you'll only find builtin modules under /sys/module/ if it has any module parameters, otherwise you won't find it there. As Masahiro already mentioned, if a builtin module has any parameters, they would be accessible under /sys/module/. (Looks like currently only module_param and MODULE_VERSION) This patch is not exactly the same, but I see a kind of overwrap. I'd like to be sure if we want this new scheme. The /sys/module is only for running kernel. One of my use cases is to create an initrd for a new kernel. > Information about built-in modules is necessary in the following cases: > > 1. When it is necessary to find out what additional parameters can be > passed to the kernel at boot time. Actually, /sys/module//parameters/ exposes this information. Doesn't it work for your purpose? No, since creating an initrd needs to know all the modalias before I get the sysfs for new kernel. Also there are no modalias at all. > 2. When you need to know which module names and their aliases are in > the kernel. This is very useful for creating an initrd image. > Hm, I do see one possible additional use-case for preserving module alias information for built-in modules - modprobe will currently error (I think, correct me if I'm wrong) if we try invoking modprobe with an alias of a built-in module, simply because this information is not in modules.builtin or modules.alias. Since kbuild already outputs modules.builtin, I would suggest outputting something like a modules.builtin.alias file (and I guess maybe a modules.builtin.param file too if that's deemed useful), in a format that is consumable by kmod/modprobe, so that modprobing an alias of a built-in module doesn't produce an error. I think this should be easy to do if we keep and parse the resulting .modinfo for builtin modules. This is just an idea, opinions welcome. I've added Lucas to CC in case he has any thoughts. Thanks, Jessica
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Wed, Mar 27, 2019 at 04:40:25PM +0100, Jessica Yu wrote: > +++ Alexey Gladkov [26/03/19 18:24 +0100]: > >On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: > >> Hi. > >> > >> (added some people to CC) > > (Thanks Masahiro for the CC!) > > >> > >> On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov > >> wrote: > >> > > >> > Problem: > >> > > >> > When a kernel module is compiled as a separate module, some important > >> > information about the kernel module is available via .modinfo section of > >> > the module. In contrast, when the kernel module is compiled into the > >> > kernel, that information is not available. > >> > >> > >> I might be missing something, but > >> vmlinux provides info of builtin modules > >> in /sys/module/. > > > >No. There are definitely not all modules. I have a builtin sha256_generic, > >but I can't find him in the /sys/module. > > Yeah, you'll only find builtin modules under /sys/module/ if it has any module > parameters, otherwise you won't find it there. As Masahiro already mentioned, > if a builtin module has any parameters, they would be accessible under > /sys/module/. > > >> (Looks like currently only module_param and MODULE_VERSION) > >> > >> This patch is not exactly the same, but I see a kind of overwrap. > >> I'd like to be sure if we want this new scheme. > > > >The /sys/module is only for running kernel. One of my use cases is > >to create an initrd for a new kernel. > > > >> > >> > Information about built-in modules is necessary in the following cases: > >> > > >> > 1. When it is necessary to find out what additional parameters can be > >> > passed to the kernel at boot time. > >> > >> > >> Actually, /sys/module//parameters/ > >> exposes this information. > >> > >> Doesn't it work for your purpose? > > > >No, since creating an initrd needs to know all the modalias before > >I get the sysfs for new kernel. Also there are no modalias at all. > > > >> > 2. When you need to know which module names and their aliases are in > >> > the kernel. This is very useful for creating an initrd image. > >> > > > Hm, I do see one possible additional use-case for preserving module alias > information for built-in modules - modprobe will currently error (I think, > correct me if I'm wrong) if we try invoking modprobe with an alias of a > built-in module, simply because this information is not in modules.builtin or > modules.alias. Yes. Patch for modprobe in my todo list. The reason I didn’t do it was because I wasn’t sure that the file format was final. > Since kbuild already outputs modules.builtin, I would suggest outputting > something like a modules.builtin.alias file (and I guess maybe a > modules.builtin.param > file too if that's deemed useful), in a format that is consumable by > kmod/modprobe, > so that modprobing an alias of a built-in module doesn't produce an error. I > think this should be easy to do if we keep and parse the resulting .modinfo > for > builtin modules. This is just an idea, opinions welcome. I've added Lucas to > CC > in case he has any thoughts. You don't like kernel.builtin.modinfo ? It is much easier to create and it has almost the same format as the modules. So I think it will be easier to parse in kmod. -- Rgrds, legion
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
Hi. Sorry for the delay. I added some comments from the build system point of view. On Wed, Mar 27, 2019 at 2:24 AM Alexey Gladkov wrote: > > On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: > > Hi. > > > > (added some people to CC) > > > > > > On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov > > wrote: > > > > > > Problem: > > > > > > When a kernel module is compiled as a separate module, some important > > > information about the kernel module is available via .modinfo section of > > > the module. In contrast, when the kernel module is compiled into the > > > kernel, that information is not available. > > > > > > I might be missing something, but > > vmlinux provides info of builtin modules > > in /sys/module/. > > No. There are definitely not all modules. I have a builtin sha256_generic, > but I can't find him in the /sys/module. > > > (Looks like currently only module_param and MODULE_VERSION) > > > > This patch is not exactly the same, but I see a kind of overwrap. > > I'd like to be sure if we want this new scheme. > > The /sys/module is only for running kernel. One of my use cases is > to create an initrd for a new kernel. > > > > > > Information about built-in modules is necessary in the following cases: > > > > > > 1. When it is necessary to find out what additional parameters can be > > > passed to the kernel at boot time. > > > > > > Actually, /sys/module//parameters/ > > exposes this information. > > > > Doesn't it work for your purpose? > > No, since creating an initrd needs to know all the modalias before > I get the sysfs for new kernel. Also there are no modalias at all. > > > > 2. When you need to know which module names and their aliases are in > > > the kernel. This is very useful for creating an initrd image. > > > > > > Proposal: > > > > > > The proposed patch does not remove .modinfo section with module > > > information from the vmlinux at the build time and saves it into a > > > separate file after kernel linking. So, the kernel does not increase in > > > size and no additional information remains in it. Information is stored > > > in the same format as in the separate modules (null-terminated string > > > array). Because the .modinfo section is already exported with a separate > > > modules, we are not creating a new API. > > > > > > It can be easily read in the userspace: > > > > > > $ tr '\0' '\n' < kernel.builtin.modinfo > > > ext4.softdep=pre: crc32c > > > ext4.license=GPL > > > ext4.description=Fourth Extended Filesystem > > > ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, > > > Theodore Ts'o and others > > > ext4.alias=fs-ext4 > > > ext4.alias=ext3 > > > ext4.alias=fs-ext3 > > > ext4.alias=ext2 > > > ext4.alias=fs-ext2 > > > md_mod.alias=block-major-9-* > > > md_mod.alias=md > > > md_mod.description=MD RAID framework > > > md_mod.license=GPL > > > md_mod.parmtype=create_on_open:bool > > > md_mod.parmtype=start_dirty_degraded:int > > > ... > > > > > > Co-Developed-by: Gleb Fotengauer-Malinovskiy > > > Signed-off-by: Gleb Fotengauer-Malinovskiy > > > Signed-off-by: Alexey Gladkov > > > --- > > > Makefile| 1 + > > > include/linux/moduleparam.h | 12 +--- > > > scripts/link-vmlinux.sh | 8 > > > 3 files changed, 14 insertions(+), 7 deletions(-) > > > > > > diff --git a/Makefile b/Makefile > > > index d5713e7b1e50..971102194c92 100644 > > > --- a/Makefile > > > +++ b/Makefile > > > @@ -1288,6 +1288,7 @@ _modinst_: > > > fi > > > @cp -f $(objtree)/modules.order $(MODLIB)/ > > > @cp -f $(objtree)/modules.builtin $(MODLIB)/ > > > + @cp -f $(objtree)/kernel.builtin.modinfo $(MODLIB)/ This file should be cleaned up by 'make clean'. Please add CLEAN_FILES around line 1336. # Directories & files removed with 'make clean' CLEAN_DIRS += $(MODVERDIR) include/ksym CLEAN_FILES += ... Also, shouldn't it be added to .gitignore? > > > $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst > > > > > > # This depmod is only for convenience to give the initial > > > diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h > > > index ba36506db4fb..5ba250d9172a 100644 > > > --- a/include/linux/moduleparam.h > > > +++ b/include/linux/moduleparam.h > > > @@ -10,23 +10,21 @@ > > > module name. */ > > > #ifdef MODULE > > > #define MODULE_PARAM_PREFIX /* empty */ > > > +#define __MODULE_INFO_PREFIX /* empty */ > > > #else > > > #define MODULE_PARAM_PREFIX KBUILD_MODNAME "." > > > +/* We cannot use MODULE_PARAM_PREFIX because some modules override it. */ > > > +#define __MODULE_INFO_PREFIX KBUILD_MODNAME "." > > > #endif > > > > > > /* Chosen so that structs with an unsigned long line up. */ > > > #define MAX_PARAM_PREFIX_LEN (64 - sizeof(unsigned long)) > > > > > > -#ifdef MODULE > > > #define __MODULE_INFO(tag, name, info) > > > \ > > > static const char __UNIQUE_ID(name)[]
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Wed, Apr 3, 2019 at 7:45 PM Masahiro Yamada wrote: > > Hi. > > Sorry for the delay. > I added some comments from the > build system point of view. > > > On Wed, Mar 27, 2019 at 2:24 AM Alexey Gladkov > wrote: > > > > On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: > > > Hi. > > > > > > (added some people to CC) > > > > > > > > > On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov > > > wrote: > > > > > > > > Problem: > > > > > > > > When a kernel module is compiled as a separate module, some important > > > > information about the kernel module is available via .modinfo section of > > > > the module. In contrast, when the kernel module is compiled into the > > > > kernel, that information is not available. > > > > > > > > > I might be missing something, but > > > vmlinux provides info of builtin modules > > > in /sys/module/. > > > > No. There are definitely not all modules. I have a builtin sha256_generic, > > but I can't find him in the /sys/module. > > > > > (Looks like currently only module_param and MODULE_VERSION) > > > > > > This patch is not exactly the same, but I see a kind of overwrap. > > > I'd like to be sure if we want this new scheme. > > > > The /sys/module is only for running kernel. One of my use cases is > > to create an initrd for a new kernel. > > > > > > > > > Information about built-in modules is necessary in the following cases: > > > > > > > > 1. When it is necessary to find out what additional parameters can be > > > > passed to the kernel at boot time. > > > > > > > > > Actually, /sys/module//parameters/ > > > exposes this information. > > > > > > Doesn't it work for your purpose? > > > > No, since creating an initrd needs to know all the modalias before > > I get the sysfs for new kernel. Also there are no modalias at all. > > > > > > 2. When you need to know which module names and their aliases are in > > > > the kernel. This is very useful for creating an initrd image. > > > > > > > > Proposal: > > > > > > > > The proposed patch does not remove .modinfo section with module > > > > information from the vmlinux at the build time and saves it into a > > > > separate file after kernel linking. So, the kernel does not increase in > > > > size and no additional information remains in it. Information is stored > > > > in the same format as in the separate modules (null-terminated string > > > > array). Because the .modinfo section is already exported with a separate > > > > modules, we are not creating a new API. > > > > > > > > It can be easily read in the userspace: > > > > > > > > $ tr '\0' '\n' < kernel.builtin.modinfo > > > > ext4.softdep=pre: crc32c > > > > ext4.license=GPL > > > > ext4.description=Fourth Extended Filesystem > > > > ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, > > > > Theodore Ts'o and others > > > > ext4.alias=fs-ext4 > > > > ext4.alias=ext3 > > > > ext4.alias=fs-ext3 > > > > ext4.alias=ext2 > > > > ext4.alias=fs-ext2 > > > > md_mod.alias=block-major-9-* > > > > md_mod.alias=md > > > > md_mod.description=MD RAID framework > > > > md_mod.license=GPL > > > > md_mod.parmtype=create_on_open:bool > > > > md_mod.parmtype=start_dirty_degraded:int > > > > ... > > > > > > > > Co-Developed-by: Gleb Fotengauer-Malinovskiy > > > > Signed-off-by: Gleb Fotengauer-Malinovskiy > > > > Signed-off-by: Alexey Gladkov > > > > --- > > > > Makefile| 1 + > > > > include/linux/moduleparam.h | 12 +--- > > > > scripts/link-vmlinux.sh | 8 > > > > 3 files changed, 14 insertions(+), 7 deletions(-) > > > > > > > > diff --git a/Makefile b/Makefile > > > > index d5713e7b1e50..971102194c92 100644 > > > > --- a/Makefile > > > > +++ b/Makefile > > > > @@ -1288,6 +1288,7 @@ _modinst_: > > > > fi > > > > @cp -f $(objtree)/modules.order $(MODLIB)/ > > > > @cp -f $(objtree)/modules.builtin $(MODLIB)/ > > > > + @cp -f $(objtree)/kernel.builtin.modinfo $(MODLIB)/ > > > This file should be cleaned up by 'make clean'. > > Please add CLEAN_FILES around line 1336. > > > # Directories & files removed with 'make clean' > CLEAN_DIRS += $(MODVERDIR) include/ksym > CLEAN_FILES += ... > > > > > Also, shouldn't it be added to .gitignore? > > > > > > > > $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst > > > > > > > > # This depmod is only for convenience to give the initial > > > > diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h > > > > index ba36506db4fb..5ba250d9172a 100644 > > > > --- a/include/linux/moduleparam.h > > > > +++ b/include/linux/moduleparam.h > > > > @@ -10,23 +10,21 @@ > > > > module name. */ > > > > #ifdef MODULE > > > > #define MODULE_PARAM_PREFIX /* empty */ > > > > +#define __MODULE_INFO_PREFIX /* empty */ > > > > #else > > > > #define MODULE_PARAM_PREFIX KBUILD_MODNAME "." > > > > +/* We cannot use MODULE_PARAM_PREFIX because some modules override it. > > > > */ > > > > +#define __MODULE_INFO_PREFIX KBUILD_MODNAME "." >
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Thu, Mar 28, 2019 at 1:04 AM Alexey Gladkov wrote: > > On Wed, Mar 27, 2019 at 04:40:25PM +0100, Jessica Yu wrote: > > +++ Alexey Gladkov [26/03/19 18:24 +0100]: > > >On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: > > >> Hi. > > >> > > >> (added some people to CC) > > > > (Thanks Masahiro for the CC!) > > > > >> > > >> On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov > > >> wrote: > > >> > > > >> > Problem: > > >> > > > >> > When a kernel module is compiled as a separate module, some important > > >> > information about the kernel module is available via .modinfo section > > >> > of > > >> > the module. In contrast, when the kernel module is compiled into the > > >> > kernel, that information is not available. > > >> > > >> > > >> I might be missing something, but > > >> vmlinux provides info of builtin modules > > >> in /sys/module/. > > > > > >No. There are definitely not all modules. I have a builtin sha256_generic, > > >but I can't find him in the /sys/module. > > > > Yeah, you'll only find builtin modules under /sys/module/ if it has any > > module > > parameters, otherwise you won't find it there. As Masahiro already > > mentioned, > > if a builtin module has any parameters, they would be accessible under > > /sys/module/. > > > > >> (Looks like currently only module_param and MODULE_VERSION) > > >> > > >> This patch is not exactly the same, but I see a kind of overwrap. > > >> I'd like to be sure if we want this new scheme. > > > > > >The /sys/module is only for running kernel. One of my use cases is > > >to create an initrd for a new kernel. > > > > > >> > > >> > Information about built-in modules is necessary in the following cases: > > >> > > > >> > 1. When it is necessary to find out what additional parameters can be > > >> > passed to the kernel at boot time. > > >> > > >> > > >> Actually, /sys/module//parameters/ > > >> exposes this information. > > >> > > >> Doesn't it work for your purpose? > > > > > >No, since creating an initrd needs to know all the modalias before > > >I get the sysfs for new kernel. Also there are no modalias at all. > > > > > >> > 2. When you need to know which module names and their aliases are in > > >> > the kernel. This is very useful for creating an initrd image. > > >> > > > > > Hm, I do see one possible additional use-case for preserving module alias > > information for built-in modules - modprobe will currently error (I think, > > correct me if I'm wrong) if we try invoking modprobe with an alias of a > > built-in module, simply because this information is not in modules.builtin > > or > > modules.alias. > > Yes. Patch for modprobe in my todo list. The reason I didn’t do it was > because I wasn’t sure that the file format was final. > > > Since kbuild already outputs modules.builtin, I would suggest outputting > > something like a modules.builtin.alias file (and I guess maybe a > > modules.builtin.param > > file too if that's deemed useful), in a format that is consumable by > > kmod/modprobe, > > so that modprobing an alias of a built-in module doesn't produce an error. I > > think this should be easy to do if we keep and parse the resulting .modinfo > > for > > builtin modules. This is just an idea, opinions welcome. I've added Lucas > > to CC > > in case he has any thoughts. > > You don't like kernel.builtin.modinfo ? Naming is often the most difficult thing. :) IMHO, 'kernel' and 'builtin' have a similar meaning here. Is 'kernel.builtin' unnecessarily too long? Perhaps, another idea is: 'builtin.alias' instead of 'modules.builtin.alias' if we want a separate file in the same format. In hindsight, 'modules.builtin' should have been 'builtin.order', I think. > It is much easier to create and it has almost the same format as the > modules. So I think it will be easier to parse in kmod. -- Best Regards Masahiro Yamada
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov wrote: > > Problem: > > When a kernel module is compiled as a separate module, some important > information about the kernel module is available via .modinfo section of > the module. In contrast, when the kernel module is compiled into the > kernel, that information is not available. > > Information about built-in modules is necessary in the following cases: > > 1. When it is necessary to find out what additional parameters can be > passed to the kernel at boot time. > > 2. When you need to know which module names and their aliases are in > the kernel. This is very useful for creating an initrd image. > > Proposal: > > The proposed patch does not remove .modinfo section with module > information from the vmlinux at the build time and saves it into a > separate file after kernel linking. So, the kernel does not increase in > size and no additional information remains in it. Information is stored > in the same format as in the separate modules (null-terminated string > array). Because the .modinfo section is already exported with a separate > modules, we are not creating a new API. > > It can be easily read in the userspace: > > $ tr '\0' '\n' < kernel.builtin.modinfo > ext4.softdep=pre: crc32c > ext4.license=GPL > ext4.description=Fourth Extended Filesystem > ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, > Theodore Ts'o and others > ext4.alias=fs-ext4 > ext4.alias=ext3 > ext4.alias=fs-ext3 > ext4.alias=ext2 > ext4.alias=fs-ext2 > md_mod.alias=block-major-9-* > md_mod.alias=md > md_mod.description=MD RAID framework > md_mod.license=GPL > md_mod.parmtype=create_on_open:bool > md_mod.parmtype=start_dirty_degraded:int > ... > > Co-Developed-by: Gleb Fotengauer-Malinovskiy > Signed-off-by: Gleb Fotengauer-Malinovskiy > Signed-off-by: Alexey Gladkov > --- I just started to take a look. I was able to compile it for x86, but got tons of warnings for powerpc. Anyway, you do not need to send v2 soon. Please give me some more time for the through review. The following is what I got by "make -j8 ARCH=powerpc CROSS_COMPILE=powerpc-linux- defconfig all" ... CC [M] drivers/scsi/st.o AR drivers/scsi/built-in.a AR drivers/built-in.a GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC init/version.o AR init/built-in.a LD vmlinux.o MODPOST vmlinux.o powerpc-linux-ld: warning: orphan section `.gnu.hash' from `linker stubs' being placed in section `.gnu.hash'. powerpc-linux-ld: warning: orphan section `.modinfo' from `arch/powerpc/platforms/pseries/pseries_energy.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `arch/powerpc/platforms/pseries/cmm.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `arch/powerpc/platforms/pasemi/gpio_mdio.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/workqueue.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/printk/printk.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/irq/spurious.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/rcu/update.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/rcu/srcutree.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/rcu/tree.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/module.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `kernel/configs.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/binfmt_script.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/binfmt_elf.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/compat_binfmt_elf.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/mbcache.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/nfs_common/nfsacl.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/nfs_common/grace.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/ext4/super.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/ext2/super.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/jbd2/journal.o' being placed in section `.modinfo'. powerpc-linux-ld: warning: orphan section `.modinfo' from `fs/fat/inode.o
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
Hi. (added some people to CC) On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov wrote: > > Problem: > > When a kernel module is compiled as a separate module, some important > information about the kernel module is available via .modinfo section of > the module. In contrast, when the kernel module is compiled into the > kernel, that information is not available. I might be missing something, but vmlinux provides info of builtin modules in /sys/module/. (Looks like currently only module_param and MODULE_VERSION) This patch is not exactly the same, but I see a kind of overwrap. I'd like to be sure if we want this new scheme. > Information about built-in modules is necessary in the following cases: > > 1. When it is necessary to find out what additional parameters can be > passed to the kernel at boot time. Actually, /sys/module//parameters/ exposes this information. Doesn't it work for your purpose? > 2. When you need to know which module names and their aliases are in > the kernel. This is very useful for creating an initrd image. > > Proposal: > > The proposed patch does not remove .modinfo section with module > information from the vmlinux at the build time and saves it into a > separate file after kernel linking. So, the kernel does not increase in > size and no additional information remains in it. Information is stored > in the same format as in the separate modules (null-terminated string > array). Because the .modinfo section is already exported with a separate > modules, we are not creating a new API. > > It can be easily read in the userspace: > > $ tr '\0' '\n' < kernel.builtin.modinfo > ext4.softdep=pre: crc32c > ext4.license=GPL > ext4.description=Fourth Extended Filesystem > ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, > Theodore Ts'o and others > ext4.alias=fs-ext4 > ext4.alias=ext3 > ext4.alias=fs-ext3 > ext4.alias=ext2 > ext4.alias=fs-ext2 > md_mod.alias=block-major-9-* > md_mod.alias=md > md_mod.description=MD RAID framework > md_mod.license=GPL > md_mod.parmtype=create_on_open:bool > md_mod.parmtype=start_dirty_degraded:int > ... > > Co-Developed-by: Gleb Fotengauer-Malinovskiy > Signed-off-by: Gleb Fotengauer-Malinovskiy > Signed-off-by: Alexey Gladkov > --- > Makefile| 1 + > include/linux/moduleparam.h | 12 +--- > scripts/link-vmlinux.sh | 8 > 3 files changed, 14 insertions(+), 7 deletions(-) > > diff --git a/Makefile b/Makefile > index d5713e7b1e50..971102194c92 100644 > --- a/Makefile > +++ b/Makefile > @@ -1288,6 +1288,7 @@ _modinst_: > fi > @cp -f $(objtree)/modules.order $(MODLIB)/ > @cp -f $(objtree)/modules.builtin $(MODLIB)/ > + @cp -f $(objtree)/kernel.builtin.modinfo $(MODLIB)/ > $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modinst > > # This depmod is only for convenience to give the initial > diff --git a/include/linux/moduleparam.h b/include/linux/moduleparam.h > index ba36506db4fb..5ba250d9172a 100644 > --- a/include/linux/moduleparam.h > +++ b/include/linux/moduleparam.h > @@ -10,23 +10,21 @@ > module name. */ > #ifdef MODULE > #define MODULE_PARAM_PREFIX /* empty */ > +#define __MODULE_INFO_PREFIX /* empty */ > #else > #define MODULE_PARAM_PREFIX KBUILD_MODNAME "." > +/* We cannot use MODULE_PARAM_PREFIX because some modules override it. */ > +#define __MODULE_INFO_PREFIX KBUILD_MODNAME "." > #endif > > /* Chosen so that structs with an unsigned long line up. */ > #define MAX_PARAM_PREFIX_LEN (64 - sizeof(unsigned long)) > > -#ifdef MODULE > #define __MODULE_INFO(tag, name, info) \ > static const char __UNIQUE_ID(name)[]\ >__used __attribute__((section(".modinfo"), unused, aligned(1)))\ > - = __stringify(tag) "=" info > -#else /* !MODULE */ > -/* This struct is here for syntactic coherency, it is not used */ > -#define __MODULE_INFO(tag, name, info) \ > - struct __UNIQUE_ID(name) {} > -#endif > + = __MODULE_INFO_PREFIX __stringify(tag) "=" info > + > #define __MODULE_PARM_TYPE(name, _type) > \ >__MODULE_INFO(parmtype, name##type, #name ":" _type) > > diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh > index c8cf45362bd6..399d7e4d11ec 100755 > --- a/scripts/link-vmlinux.sh > +++ b/scripts/link-vmlinux.sh > @@ -258,10 +258,12 @@ if [ -n "${CONFIG_KALLSYMS}" ]; then > > # step 1 > vmlinux_link "" .tmp_vmlinux1 > + "${OBJCOPY}" -R .modinfo .tmp_vmlinux1 > kallsyms .tmp_vmlinux1 .tmp_kallsyms1.o > > # step 2 > vmlinux_link .tmp_kallsyms1.o .tmp_vmlinux2 > + "${OBJCOPY}" -R .modinfo .tmp_vmlinux2 > kallsyms .tmp_vmlinux2 .tmp_kallsyms2.o > > # step 3 > @@ -273,6 +275,7 @@ if [ -n "${CONFIG_KALLSYMS}" ]; then > kallsyms_vmlinux=.tmp_vmlinux3 > > vmlinux_link .tmp_ka
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Wed, Mar 27, 2019 at 8:40 AM Jessica Yu wrote: > >No. There are definitely not all modules. I have a builtin sha256_generic, > >but I can't find him in the /sys/module. > > Yeah, you'll only find builtin modules under /sys/module/ if it has any module > parameters, otherwise you won't find it there. As Masahiro already mentioned, > if a builtin module has any parameters, they would be accessible under > /sys/module/. Could we please change that and add the sysfs entry regardless of what's being discussed here? Not having the entry there simply because we don't have parameters for that module always annoyed me. Lucas De Marchi
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Wed, Mar 27, 2019 at 9:04 AM Alexey Gladkov wrote: > > On Wed, Mar 27, 2019 at 04:40:25PM +0100, Jessica Yu wrote: > > +++ Alexey Gladkov [26/03/19 18:24 +0100]: > > >On Fri, Mar 22, 2019 at 02:34:12PM +0900, Masahiro Yamada wrote: > > >> Hi. > > >> > > >> (added some people to CC) > > > > (Thanks Masahiro for the CC!) > > > > >> > > >> On Fri, Mar 15, 2019 at 7:10 PM Alexey Gladkov > > >> wrote: > > >> > > > >> > Problem: > > >> > > > >> > When a kernel module is compiled as a separate module, some important > > >> > information about the kernel module is available via .modinfo section > > >> > of > > >> > the module. In contrast, when the kernel module is compiled into the > > >> > kernel, that information is not available. > > >> > > >> > > >> I might be missing something, but > > >> vmlinux provides info of builtin modules > > >> in /sys/module/. > > > > > >No. There are definitely not all modules. I have a builtin sha256_generic, > > >but I can't find him in the /sys/module. > > > > Yeah, you'll only find builtin modules under /sys/module/ if it has any > > module > > parameters, otherwise you won't find it there. As Masahiro already > > mentioned, > > if a builtin module has any parameters, they would be accessible under > > /sys/module/. > > > > >> (Looks like currently only module_param and MODULE_VERSION) > > >> > > >> This patch is not exactly the same, but I see a kind of overwrap. > > >> I'd like to be sure if we want this new scheme. > > > > > >The /sys/module is only for running kernel. One of my use cases is > > >to create an initrd for a new kernel. > > > > > >> > > >> > Information about built-in modules is necessary in the following cases: > > >> > > > >> > 1. When it is necessary to find out what additional parameters can be > > >> > passed to the kernel at boot time. > > >> > > >> > > >> Actually, /sys/module//parameters/ > > >> exposes this information. > > >> > > >> Doesn't it work for your purpose? > > > > > >No, since creating an initrd needs to know all the modalias before > > >I get the sysfs for new kernel. Also there are no modalias at all. > > > > > >> > 2. When you need to know which module names and their aliases are in > > >> > the kernel. This is very useful for creating an initrd image. > > >> > > > > > Hm, I do see one possible additional use-case for preserving module alias > > information for built-in modules - modprobe will currently error (I think, > > correct me if I'm wrong) if we try invoking modprobe with an alias of a > > built-in module, simply because this information is not in modules.builtin > > or > > modules.alias. > > Yes. Patch for modprobe in my todo list. The reason I didn’t do it was > because I wasn’t sure that the file format was final. > > > Since kbuild already outputs modules.builtin, I would suggest outputting > > something like a modules.builtin.alias file (and I guess maybe a > > modules.builtin.param > > file too if that's deemed useful), in a format that is consumable by > > kmod/modprobe, > > so that modprobing an alias of a built-in module doesn't produce an error. I > > think this should be easy to do if we keep and parse the resulting .modinfo > > for > > builtin modules. This is just an idea, opinions welcome. I've added Lucas > > to CC > > in case he has any thoughts. > > You don't like kernel.builtin.modinfo ? > > It is much easier to create and it has almost the same format as the > modules. So I think it will be easier to parse in kmod. adding a modules.builtin.alias with the same format of modules.alias means during modprobe we would only need to load one more file to lookup aliases. That doesn't mean the kernel built system should do it though. The same way it's depmod job to create the modules.alias{,.bin}, we could leave this to depmod if it's in fact useful to split the information. I think your version is simple enough and we would get more information that would be useful for modinfo. It would indeed be nice to output something useful in "modinfo ext4". In kmod, I think we would create modules.builtin.modinfo{,.bin} and just add the aliases to modules.alias{,.bin}, This would keep the names consistent with what is already there. thanks Lucas De Marchi > > -- > Rgrds, legion >
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Thu, Mar 28, 2019 at 10:41:59AM -0700, Lucas De Marchi wrote: > On Wed, Mar 27, 2019 at 8:40 AM Jessica Yu wrote: > > >No. There are definitely not all modules. I have a builtin sha256_generic, > > >but I can't find him in the /sys/module. > > > > Yeah, you'll only find builtin modules under /sys/module/ if it has any > > module > > parameters, otherwise you won't find it there. As Masahiro already > > mentioned, > > if a builtin module has any parameters, they would be accessible under > > /sys/module/. > > Could we please change that and add the sysfs entry regardless of > what's being discussed here? Not having the entry there simply because > we don't have parameters for that module always annoyed me. What is the sysfs directory going to show? Will it just be empty? Feel free to send a patch for this, but from what I remember, it wasn't the easiest thing to do for some reason. But given that the code was implemented before git was, I can't quite remember. thanks, greg k-h
Re: [RESEND PATCH v1] moduleparam: Save information about built-in modules in separate file
On Thu, Mar 28, 2019 at 11:45 AM Greg KH wrote: > > On Thu, Mar 28, 2019 at 10:41:59AM -0700, Lucas De Marchi wrote: > > On Wed, Mar 27, 2019 at 8:40 AM Jessica Yu wrote: > > > >No. There are definitely not all modules. I have a builtin > > > >sha256_generic, > > > >but I can't find him in the /sys/module. > > > > > > Yeah, you'll only find builtin modules under /sys/module/ if it has any > > > module > > > parameters, otherwise you won't find it there. As Masahiro already > > > mentioned, > > > if a builtin module has any parameters, they would be accessible under > > > /sys/module/. > > > > Could we please change that and add the sysfs entry regardless of > > what's being discussed here? Not having the entry there simply because > > we don't have parameters for that module always annoyed me. What is the benefit compared to wasting some memory for the directory? > > What is the sysfs directory going to show? Will it just be empty? > > Feel free to send a patch for this, but from what I remember, it wasn't > the easiest thing to do for some reason. But given that the code was > implemented before git was, I can't quite remember. I am pretty sure we allow empty attribute groups, so it is probbaly as simple as removing "If (!params) return 0;" form module_param_sysfs_setup() and making sure we always create "parameters" group instead of doing it on first visible parameter in add_sysfs_param(). I suppose we no longer want to skip over parameters that are not readable nor writable either? Thanks. -- Dmitry