[PATCH] Add 'nodoc' option
I've written this patch, and it seems to work, but I don't really know what I'm doing and I fear I may have broken something. I don't know what: required_argument = 1 optional_arguments = 4 mean, so I don't know whether I should have adjusted them when adding this new option. >From 26896bd43618e30cb3563114663ad6c39cef7dc6 Mon Sep 17 00:00:00 2001 From: Matthew Wilcox <mawil...@microsoft.com> Date: Thu, 26 Jan 2017 11:43:24 -0500 Subject: [PATCH] kerneldoc: Add :nodoc: option This is functionality the perl script already has, we just need to expose it to our rst files. --- Documentation/sphinx/kerneldoc.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index d15e07f36881..d86d88da1d75 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -47,6 +47,7 @@ class KernelDocDirective(Directive): optional_arguments = 4 option_spec = { 'doc': directives.unchanged_required, +'nodoc': directives.unchanged, 'functions': directives.unchanged_required, 'export': directives.unchanged, 'internal': directives.unchanged, @@ -74,6 +75,8 @@ class KernelDocDirective(Directive): export_file_patterns = str(self.options.get('internal')).split() elif 'doc' in self.options: cmd += ['-function', str(self.options.get('doc'))] +elif 'nodoc' in self.options: +cmd += ['-no-doc-sections'] elif 'functions' in self.options: for f in str(self.options.get('functions')).split(): cmd += ['-function', f] -- 2.11.0 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] kernel-doc: Handle returning pointers to pointers
Clearly nobody ever tried to build the documentation for the radix tree before: include/linux/radix-tree.h:400: warning: cannot understand function prototype: 'void ** radix_tree_iter_init(struct radix_tree_iter *iter, unsigned long start) ' Indeed, the regexes only handled a single '*', not one-or-more. I have tried to fix that, but now I have perl regexes all over my hands, and I fear I shall never be clean again. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- scripts/kernel-doc | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 030fc633acd4..b193b0337d1b 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -2533,21 +2533,21 @@ sub dump_function($$) { $noret = 1; } elsif ($prototype =~ m/^()([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || $prototype =~ m/^(\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || - $prototype =~ m/^(\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || + $prototype =~ m/^(\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || $prototype =~ m/^(\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || $prototype =~ m/^(\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || $prototype =~ m/^(\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || - $prototype =~ m/^(\w+\s+\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || + $prototype =~ m/^(\w+\s+\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)/ || $prototype =~ m/^()([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || $prototype =~ m/^(\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || - $prototype =~ m/^(\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || + $prototype =~ m/^(\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || $prototype =~ m/^(\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || - $prototype =~ m/^(\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || + $prototype =~ m/^(\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || $prototype =~ m/^(\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || - $prototype =~ m/^(\w+\s+\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || + $prototype =~ m/^(\w+\s+\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || $prototype =~ m/^(\w+\s+\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || - $prototype =~ m/^(\w+\s+\w+\s+\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || - $prototype =~ m/^(\w+\s+\w+\s*\*\s*\w+\s*\*\s*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/) { + $prototype =~ m/^(\w+\s+\w+\s+\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/ || + $prototype =~ m/^(\w+\s+\w+\s*\*+\s*\w+\s*\*+\s*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/) { $return_type = $1; $declaration_name = $2; my $args = $3; -- 2.11.0.301.g722e3be85.dirty -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] kernel-doc: Handle returning pointers to pointers
From: Markus Heiser [mailto:markus.hei...@darmarit.de] > Am 23.01.2017 um 09:18 schrieb Matthew Wilcox > <mawil...@microsoft.com>: > Hi Matthew ! > > short answer: Thanks a lot > Acked-by: Markus Heiser <markus.hei...@darmarit.de> Excellent! > to be more verbose: >what I have tested and what I recommend ... > > I maintain my own stack of "linuxdoc" with a python version > of the kernel-doc script (hosted on github). It uses the same > regexes as the perl version (using a python rewrite here has some > other benefits, one you will see below). I merged your patch: Are there plans to merge that? It feels so odd to have a python script calling a perl script ... > Another part of my stack is the 'sphkerneldoc' repository > with reST, generated from whole kernel source tree. Here is > the diff of what your patch produce on the whole source tree: Oh, that's funny. Other places had the same problem ... I guess nobody had tried to fix that warning yet :-) > Here is my patch for this. May you like to add this to your patch: > > diff --git a/include/linux/radix-tree.h b/include/linux/radix-tree.h > index 5dea8f6..33cbc1b 100644 > --- a/include/linux/radix-tree.h > +++ b/include/linux/radix-tree.h > @@ -164,7 +164,7 @@ static inline unsigned int iter_shift(const struct > radix_tree_iter *iter) > } > > /** > - * Radix-tree synchronization > + * DOC: Radix-tree synchronization > * > * The radix-tree API requires that users provide all synchronisation (with > * specific exceptions, noted below). > > with this small patch, we also get header's 'DOC: comment' as HTML. I have that and much, much more in my tree right now ... I wrote a radix-tree.rst on the plane, and I've been tidying up all kinds of problems that sphinx noticed, or just reading the documentation made glaringly obvious. N�r��yb�X��ǧv�^�){.n�+{�v�"��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
Omitting documentation for internal structure members
Here's a little glitch that I'd like to see fixed: struct radix_tree_iter radix tree iterator state Definition struct radix_tree_iter { unsigned long index; unsigned long next_index; unsigned long tags; struct radix_tree_node * node; #ifdef CONFIG_RADIX_TREE_MULTIORDER unsigned int shift; #endif }; I'd like to see the ifdef/endif not included, and the 'unsigned int shift' not included either. I think it's appropriate to not include any preprocessor lines in the definition of the struct (general agreement on that front?) Then I'd like a way to indicate to kernel-doc that I want to omit 'shift' from the struct definition. It's not for public use; there's an accessor to get at it, and I don't want it documented. If there isn't already a way to indicate this to kernel-doc, might I suggest this little gem ... /** * struct radix_tree_iter - radix tree iterator state * @index: index of current slot * @next_index: one beyond the last index for this chunk * @tags: bit-mask for tag-iterating * @node: node that contains current slot - @shift: shift for the node that holds our slots * * The radix tree iterator works in terms of "chunks" of slots. A chunk is a * subinterval of slots contained within one radix tree leaf node. It is * described by a pointer to its first slot and a struct radix_tree_iter * which holds the chunk's position in the tree and its size. For tagged * iteration radix_tree_iter also holds the slots' bit-mask for one chosen * radix tree tag. */ If the line starts with a '-' instead of a '*', that indicates that this definition should be omitted. We're still documenting it for internal usage, but it doesn't appear in the public documentation. Does that seem reasonable, or am I just being weird? N�r��yb�X��ǧv�^�){.n�+{�v�"��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
RE: Omitting documentation for internal structure members
From: Jani Nikula [mailto:jani.nik...@linux.intel.com] > You can use /* private: */ within the struct to indicate the following > members should not be included in the generated documentation. It does > however mean you can't then document the members either, or you'll get > warnings. > > I'm generally not too happy about the mechanism, but it's there, perhaps > not very well advertized. I don't know how I failed to see that in the documentation. I read doc-guide/kernel-doc.rst about three times this week. @@ -154,6 +157,7 @@ struct radix_tree_iter { unsigned long next_index; unsigned long tags; struct radix_tree_node *node; +/* private: Do not use directly; call iter_shift() or __set_iter_shift() */ #ifdef CONFIG_RADIX_TREE_MULTIORDER unsigned intshift; #endif That solves my problem. Thanks!
[PATCH] Improve sparse documentation
Add documentation of -DCONFIG_SPARSE_RCU_POINTER. I started to add documentation of -D__CHECK_ENDIAN__ as well, but discovered I'm too late; that's now enabled by default. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> diff --git a/Documentation/dev-tools/sparse.rst b/Documentation/dev-tools/sparse.rst index 78aa00a604a0..2f943354322c 100644 --- a/Documentation/dev-tools/sparse.rst +++ b/Documentation/dev-tools/sparse.rst @@ -103,3 +103,9 @@ recompiled, or use "make C=2" to run sparse on the files whether they need to The optional make variable CF can be used to pass arguments to sparse. The build system passes -Wbitwise to sparse automatically. + +Checking RCU annotations + + +RCU annotations are not checked by default. To enable RCU annotation +checks, include -DCONFIG_SPARSE_RCU_POINTER in your CF flags. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] kernel-doc: Handle returning pointers to pointers
From: Markus Heiser [mailto:markus.hei...@darmarit.de] > Am 23.01.2017 um 16:24 schrieb Jonathan Corbet: > > Markus, would you consider sending out a new patch set for review? > > Yes, I send RFC soon ... Could I ask for some features? I'd've been trying to add them to the perl script, but since I am terrible at writing both Perl and Python, and I have your attention right now ... 1. An option to select which functions are output by regular expression. I would like to be able to say: .. kernel-doc:: lib/radix-tree.c :functions: radix.* to avoid the IDA/IDR functions which now live in radix-tree.c from having their kernel-doc output. 2. An option to output everything *except* the /** DOC: */ comments. I want to be able to do something like this in my .rst file: --- 8< --- Locking --- .. kernel-doc:: include/linux/radix-tree.h :doc: Radix-tree synchronization The Public API == The public API can be found in . To use a radix tree in your data structure, embed a :c:type:`struct radix_tree_root` in it, and initialise it using ``INIT_RADIX_TREE``. You can also use a file-local or global radix tree by defining a :c:type:`RADIX_TREE` as you would a :c:type:`LIST_HEAD`. .. kernel-doc:: include/linux/radix-tree.h :nodoc: --- >8 --- 3. I think it would also make sense, as well as being able to ask for 'all exported symbols', to be able to ask for 'all non-static symbols'; we want to document __radix_tree_lookup(), but we only want to export it as a symbol if a modular user shows up (this isn't critical for me since I want to filter out the idr/ida functions as well, but it might be useful for other files).
Re: [PATCH] Documentation: filesystems: update filesystem locking documentation
On Mon, Jul 31, 2017 at 11:53:35AM -0400, Sean Anderson wrote: > - i_mutex(inode) > -lookup: yes > + i_rwsem(inode) > +lookup: shared > create: yes Could you change all the 'yes' to 'exclusive' when it's changed from mutex to rwsem? > link:yes (both) > mknod: yes > @@ -92,7 +92,7 @@ atomic_open:yes > tmpfile: no > > > - Additionally, ->rmdir(), ->unlink() and ->rename() have ->i_mutex on > + Additionally, ->rmdir(), ->unlink() and ->rename() have ->i_rwsem on > victim. > cross-directory ->rename() has (per-superblock) ->s_vfs_rename_sem. > > @@ -111,7 +111,7 @@ prototypes: > > locking rules: > all may block > - i_mutex(inode) > + i_rwsem(inode) > list:no > get: no > set: yes > @@ -217,7 +217,7 @@ prototypes: > locking rules: > All except set_page_dirty and freepage may block > > - PageLocked(page)i_mutex > + PageLocked(page)i_rwsem > writepage: yes, unlocks (see below) > readpage:yes, unlocks > writepages: > @@ -439,6 +439,7 @@ prototypes: > ssize_t (*read_iter) (struct kiocb *, struct iov_iter *); > ssize_t (*write_iter) (struct kiocb *, struct iov_iter *); > int (*iterate) (struct file *, struct dir_context *); > + int (*iterate_shared) (struct file *, struct dir_context *); > unsigned int (*poll) (struct file *, struct poll_table_struct *); > long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); > long (*compat_ioctl) (struct file *, unsigned int, unsigned long); > @@ -480,6 +481,10 @@ mutex or just to use i_size_read() instead. > Note: this does not protect the file->f_pos against concurrent modifications > since this is something the userspace has to take care about. > > +->iterate() is called with i_rwsem held. > + > +->iterate_shared() is called with i_rwsem shared. > + > ->fasync() is responsible for maintaining the FASYNC bit in filp->f_flags. > Most instances call fasync_helper(), which does that maintenance, so it's > not normally something one needs to worry about. Return values > 0 will be > -- > 2.13.2 > -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/4] fs/dcache: Report negative dentry number in dentry-state
On Mon, Jul 17, 2017 at 09:39:31AM -0400, Waiman Long wrote: > @@ -63,9 +63,10 @@ struct qstr { > struct dentry_stat_t { > long nr_dentry; > long nr_unused; > - long age_limit; /* age in seconds */ > - long want_pages; /* pages requested by system */ > - long dummy[2]; > + long nr_negative; /* # of negative dentries */ > + long age_limit; /* age in seconds */ > + long want_pages;/* pages requested by system */ > + long dummy; > }; > extern struct dentry_stat_t dentry_stat; You can't just insert a field in the middle like that. It'll break any code parsing /proc/sys/fs/dentry-state. You have to put it at the end: long age_limit; /* age in seconds */ long want_pages; /* pages requested by system */ - long dummy[2]; + long nr_negative; /* # of negative dentries */ + long dummy; }; -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/4] fs/dcache: Limit numbers of negative dentries
On Mon, Jul 17, 2017 at 09:39:30AM -0400, Waiman Long wrote: > The number of positive dentries is limited by the number of files > in the filesystems. The number of negative dentries, however, > has no limit other than the total amount of memory available in > the system. So a rogue application that generates a lot of negative > dentries can potentially exhaust most of the memory available in the > system impacting performance on other running applications. > > To prevent this from happening, the dcache code is now updated to limit > the amount of the negative dentries in the LRU lists that can be kept > as a percentage of total available system memory. The default is 5% > and can be changed by specifying the "neg_dentry_pc=" kernel command > line option. I see the problem, but rather than restricting the number of negative dentries to be a fraction of the total amount of memory in the machine, wouldn't it make more sense to limit the number of negative dentries to be some multiple of the number of positive dentries currently in the system? Or make negative dentries more easily prunable. For example, we could allocate them from a separate slab and use the existing reclaim mechanism to just throw them away. Since they can't be pinned by an inode, they're much easier to get rid of than positive dentries. Might make changing a dentry from positive to negative or vice versa a bit more expensive ... -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Documentation license
I've written a couple of thousand words of documentation for the XArray. I need to decide how to license it. There are no SPDX tags in Documentation/ to date, so I have no examples to crib from. 1. How does one add an SPDX tag to an rst file? 2. What license should I use? I'd like people to be able to produce dead tree versions of the documentation, but if they make improvements to it, I want to see them. Something in the CC BY-SA 4.0 vein? 3. Is there a problem with extracting kernel-doc from a GPLv2 licensed file and then redistributing the result under CC BY-SA? They seem to have the same broad intent, but lawyers may grumble. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Check all .c files for bad kernel-doc comments
On Mon, Oct 30, 2017 at 12:40:20PM +0900, Masahiro Yamada wrote: > 2017-10-28 4:41 GMT+09:00 Matthew Wilcox <wi...@infradead.org>: > > Implement a '-none' output mode for kernel-doc which will only output > > warning messages, and suppresses the warning message about there being > > no kernel-doc in the file. Add it to the rule to build .o files from .c > > files, so it will check all .c files that have been modified. > > > > Adds about 1300 warnings to my build, but will hopefully discourage > > people from introducing more kerneldoc mistakes. > > Basically, I think this is good, > but it is controversial to sprinkle warnings by default. Yes, it is. I just got three nastygrams from 01.org ;-) But if it's not turned on by default, then people aren't going to notice when they introduce new warnings. I think it needs to be up to someone like Andrew or Linus to decide when to add these warnings by default. Maybe we should do something like you have below for now, then work on cleaning up a few hundred of these warnings, then enable this by default? Thanks for looking at this! > Maybe, > > ifeq ($(KBUILD_ENABLE_EXTRA_GCC_CHECKS),) > cmd_checkdoc = $(srctree)/scripts/kernel-doc -none $< ; > endif > > > so that this is checked only when W=... is given? -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Check all .c files for bad kernel-doc comments
On Mon, Oct 30, 2017 at 05:19:18PM +0200, Jani Nikula wrote: > Related, there was also a script to do reStructuredText lint style > checks in addition to the kernel-doc checks using make CHECK and > C=1. See http://mid.mail-archive.com/87h98quc1w.fsf@intel.com I don't really care which patch goes in. If I understand your python script correctly, it relies on having various python packages installed. Unless we're going to switch kernel-doc over to being written in python, I'd prefer to not require additional dependencies. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] documentation: core-api: rearrange a few kernel-api chapters and sections
On Thu, Apr 26, 2018 at 06:11:02PM -0700, Randy Dunlap wrote: > Rearrange some kernel-api chapters and sections to group them > together better. > > - move Bit Operations from Basic C Library Functions to Basic > Kernel Library Functions (now adjacent to Bitmap Operations since > they are not typical C library functions) > > - move Sorting from Math Functions to Basic Kernel Library Functions > since sort functions are more Basic than Math Functions > > - move Text Searching from Math Functions to Basic Kernel Library > Functions (keep Sorting and Searching close to each other) > > - combine CRC and Math functions together into the (newly named) > CRC and Math Functions chapter > > Signed-off-by: Randy Dunlap <rdun...@infradead.org> This all makes sense to me. Acked-by: Matthew Wilcox <mawil...@microsoft.com> -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks
On Thu, May 10, 2018 at 10:38:16AM -0400, Mauro Carvalho Chehab wrote: > Comments that end with a comma and have certain keywords > should be presented as code-blocks I think this is a bit fragile. Why not just search for ':\n'? Is there ever a case where we want to write: /** * foo is a bar: * wibble */ and have wibble not be a code-block? -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/5] PCI/AER: Define and allocate aer_stats structure for AER capable devices
On Wed, May 23, 2018 at 10:20:10AM -0400, Jes Sorensen wrote: > > +++ b/drivers/pci/pcie/aer/aerdrv_stats.c > > @@ -0,0 +1,64 @@ > > +// SPDX-License-Identifier: GPL-2.0 > > Fix the formatting please - that gross // gibberish doesn't belong there. Sorry, Jes. The Chief Penguin has Spoken, and that's the preferred syntax: 2. Style: The SPDX license identifier is added in form of a comment. The comment style depends on the file type:: C source: // SPDX-License-Identifier: (you can dig up the discussion around this on the mailing list if you like. Linus actually thinks that C++ single-line comments are one of the few things that language got right) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] Documentation/sphinx: add "nodocs" directive
On Mon, Jun 18, 2018 at 04:36:34PM +0300, Mike Rapoport wrote: > Hi, > > These patches allow passing "-no-doc-sections" option to scripts/kernel-doc > from the sphinx generator. > > This allows to avoid duplicated DOC: sections when "kernel-doc:" directive > is used without explicit selection of functions or function types. For > instance, [1] has "IDA description" and "idr synchronization" twice. Hah, I just found an abandoned patch for this in a disused git tree. I was wondering whether I needed to resurrect it. Enthusiastically, Acked-by: Matthew Wilcox -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] Documentation/sphinx: add "nodocs" directive
On Mon, Jun 18, 2018 at 10:10:28AM -0700, Matthew Wilcox wrote: > On Mon, Jun 18, 2018 at 04:36:34PM +0300, Mike Rapoport wrote: > > Hi, > > > > These patches allow passing "-no-doc-sections" option to scripts/kernel-doc > > from the sphinx generator. > > > > This allows to avoid duplicated DOC: sections when "kernel-doc:" directive > > is used without explicit selection of functions or function types. For > > instance, [1] has "IDA description" and "idr synchronization" twice. > > Hah, I just found an abandoned patch for this in a disused git tree. > I was wondering whether I needed to resurrect it. Enthusiastically, > > Acked-by: Matthew Wilcox Here's the patch I found (I couldn't refind it at the time): diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index d15e07f36881..d86d88da1d75 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -47,6 +47,7 @@ class KernelDocDirective(Directive): optional_arguments = 4 option_spec = { 'doc': directives.unchanged_required, +'nodoc': directives.unchanged, 'functions': directives.unchanged_required, 'export': directives.unchanged, 'internal': directives.unchanged, @@ -74,6 +75,8 @@ class KernelDocDirective(Directive): export_file_patterns = str(self.options.get('internal')).split() elif 'doc' in self.options: cmd += ['-function', str(self.options.get('doc'))] +elif 'nodoc' in self.options: +cmd += ['-no-doc-sections'] elif 'functions' in self.options: for f in str(self.options.get('functions')).split(): cmd += ['-function', f] I did it while I was trying to create good radix tree documentation, which led to me realising that was a Herculean task (specifically: the stables). I ended up doing this instead: +The Public API +== + +The public API can be found in . To use a +radix tree in your data structure, embed a :c:type:`struct radix_tree_root` +in it, and initialise it using ``INIT_RADIX_TREE``. You can also use +a file-local or global radix tree by defining a :c:type:`RADIX_TREE` as you +would a :c:type:`LIST_HEAD`. + +.. Not actually "internal", but I need to exclude the 'doc' paragraph, and + this is the best way to do it. +.. kernel-doc:: include/linux/radix-tree.h + :internal: + +.. kernel-doc:: lib/radix-tree.c + :export: I'm not sure if I agree with me-of-January-2017 that this is the "best" way to do it, but maybe that'll point to another way of achieving the same thing. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 1/2] Documentation/sphinx: allow "functions" with no parameters
On Sat, Jun 30, 2018 at 12:05:10AM +0300, Mike Rapoport wrote: > @@ -488,14 +488,19 @@ doc: *title* > .. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c > :doc: High Definition Audio over HDMI and Display Port > > -functions: *function* *[...]* > +functions: *[ function ...]* >Include documentation for each *function* in *source*. > + If no *function* if specified, the documentaion for all functions "is specified". "documentation". > - Example:: > + Examples:: > > .. kernel-doc:: lib/bitmap.c > :functions: bitmap_parselist bitmap_parselist_user > > +.. kernel-doc:: lib/idr.c > + :functions: Is this the right syntax? Should we rather have: .. kernel-doc:: lib/idr.c :functions: * -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Check all .c files for bad kernel-doc comments
From: Matthew Wilcox <mawil...@microsoft.com> Implement a '-none' output mode for kernel-doc which will only output warning messages, and suppresses the warning message about there being no kernel-doc in the file. Add it to the rule to build .o files from .c files, so it will check all .c files that have been modified. Adds about 1300 warnings to my build, but will hopefully discourage people from introducing more kerneldoc mistakes. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- scripts/Makefile.build | 3 +++ scripts/kernel-doc | 25 - 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/scripts/Makefile.build b/scripts/Makefile.build index 061d0c3a420a..f0f907af53c7 100644 --- a/scripts/Makefile.build +++ b/scripts/Makefile.build @@ -108,6 +108,8 @@ ifneq ($(KBUILD_CHECKSRC),0) endif endif +cmd_checkdoc = $(srctree)/scripts/kernel-doc -none $< ; + # Do section mismatch analysis for each module/built-in.o ifdef CONFIG_DEBUG_SECTION_MISMATCH cmd_secanalysis = ; scripts/mod/modpost $@ @@ -291,6 +293,7 @@ define rule_cc_o_c $(call echo-cmd,checksrc) $(cmd_checksrc) \ $(call cmd_and_fixdep,cc_o_c) \ $(cmd_modversions_c) \ + $(cmd_checkdoc) \ $(call echo-cmd,objtool) $(cmd_objtool) \ $(call echo-cmd,record_mcount) $(cmd_record_mcount) endef diff --git a/scripts/kernel-doc b/scripts/kernel-doc index 9d3eafea58f0..c69583440a44 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -58,6 +58,7 @@ Output format selection (mutually exclusive): -man Output troff manual page format. This is the default. -rst Output reStructuredText format. -textOutput plain text format. + -noneDo not output documentation, only warnings. Output selection (mutually exclusive): -export Only output documentation for symbols that have been @@ -532,6 +533,8 @@ while ($ARGV[0] =~ m/^-(.*)/) { $output_mode = "gnome"; @highlights = @highlights_gnome; $blankline = $blankline_gnome; +} elsif ($cmd eq "-none") { + $output_mode = "none"; } elsif ($cmd eq "-module") { # not needed for XML, inherits from calling document $modulename = shift @ARGV; } elsif ($cmd eq "-function") { # to only output specific functions @@ -2117,6 +2120,24 @@ sub output_blockhead_list(%) { } } + +## none mode output functions + +sub output_function_none(%) { +} + +sub output_enum_none(%) { +} + +sub output_typedef_none(%) { +} + +sub output_struct_none(%) { +} + +sub output_blockhead_none(%) { +} + ## # generic output function for all types (function, struct/union, typedef, enum); # calls the generated, variable output_ function name based on @@ -3136,7 +3157,9 @@ sub process_file($) { } } if ($initial_section_counter == $section_counter) { - print STDERR "${file}:1: warning: no structured comments found\n"; + if ($output_mode ne "none") { + print STDERR "${file}:1: warning: no structured comments found\n"; + } if (($output_selection == OUTPUT_INCLUDE) && ($show_not_found == 1)) { print STDERR "Was looking for '$_'.\n" for keys %function_table; } -- 2.11.0.301.g722e3be85.dirty -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/2] fs: Fix attr.c kernel-doc
A couple of minor warnings. Signed-off-by: Matthew Wilcox --- fs/attr.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/fs/attr.c b/fs/attr.c index e3d53bf12240..d22e8187477f 100644 --- a/fs/attr.c +++ b/fs/attr.c @@ -120,7 +120,6 @@ EXPORT_SYMBOL(setattr_prepare); * inode_newsize_ok - may this inode be truncated to a given size * @inode: the inode to be truncated * @offset:the new size to assign to the inode - * @Returns: 0 on success, -ve errno on failure * * inode_newsize_ok must be called with i_mutex held. * @@ -130,6 +129,8 @@ EXPORT_SYMBOL(setattr_prepare); * returned. @inode must be a file (not directory), with appropriate * permissions to allow truncate (inode_newsize_ok does NOT check these * conditions). + * + * Return: 0 on success, -ve errno on failure */ int inode_newsize_ok(const struct inode *inode, loff_t offset) { @@ -205,7 +206,7 @@ EXPORT_SYMBOL(setattr_copy); /** * notify_change - modify attributes of a filesytem object * @dentry:object affected - * @iattr: new attributes + * @attr: new attributes * @delegated_inode: returns inode, if the inode is delegated * * The caller must hold the i_mutex on the affected object. -- 2.18.0 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2] fs: Add more kernel-doc to the produced documentation
People have gone to all the effort of writing kernel-doc for these functions; the least we can do is put them in the "Other functions" part of the VFS documentation. Signed-off-by: Matthew Wilcox --- Documentation/filesystems/index.rst | 33 + 1 file changed, 33 insertions(+) diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst index 53b89d0edc15..46d1b1be3a51 100644 --- a/Documentation/filesystems/index.rst +++ b/Documentation/filesystems/index.rst @@ -71,6 +71,39 @@ Other Functions .. kernel-doc:: fs/block_dev.c :export: +.. kernel-doc:: fs/anon_inodes.c + :export: + +.. kernel-doc:: fs/attr.c + :export: + +.. kernel-doc:: fs/d_path.c + :export: + +.. kernel-doc:: fs/dax.c + :export: + +.. kernel-doc:: fs/direct-io.c + :export: + +.. kernel-doc:: fs/file_table.c + :export: + +.. kernel-doc:: fs/libfs.c + :export: + +.. kernel-doc:: fs/posix_acl.c + :export: + +.. kernel-doc:: fs/stat.c + :export: + +.. kernel-doc:: fs/sync.c + :export: + +.. kernel-doc:: fs/xattr.c + :export: + The proc filesystem === -- 2.18.0 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Documentation: security/credentials.rst: explain need to sort group_list
On Sat, Jan 06, 2018 at 11:09:08AM -0700, Jonathan Corbet wrote: > On Tue, 2 Jan 2018 13:04:31 -0800 > Matthew Wilcox <wi...@infradead.org> wrote: > > > > +When replacing the group list, the new list must be sorted before it > > > +is added to the credential, as a binary search is used to test for > > > +membership. In practice, this means ``groups_sort()`` should be > > > > For a .rst file, shouldn't we be using :c:func:`groups_sort` instead of > > ``groups_sort()``? > > There is value in using the c:func syntax, as it will generate > cross-references to the kerneldoc comments for those functions. In this > case, it would appear that these comments exist, but nobody has pulled > them into the docs yet. I took the liberty of adding :c:func: references > to this patch on its way into docs-next so that things will Just Work on > that happy day when somebody adds the function documentation. Thanks for making that substitution. I've been thinking about all the kernel-doc we have that's completely unincorporated. I've also been thinking about core-api/kernel-api.rst which to my mind is completely unreadable in its current form -- look at https://www.kernel.org/doc/html/latest/core-api/kernel-api.html and you wouldn't really know there's anything in it beyond the List Management Functions. I think the right path forward is to have kernel-api.rst be the dumping ground for all the files with kernel-doc but nothing more. That gives us somewhere to link to. Then we need little stories about how all the functions in a subsystem fit together. For example, we can create a list.rst which explains how this is a doubly-linked list that you use by embedding a list_head into your data structure, and has O(1) insertion/deletion, etc, etc. Then we would move all the list.h kernel-doc from kernel-api.rst into list.rst. Is this a reasonable strategy to follow? Does anyone have a better strategy? I mean ... you've written a book, you presumably have some idea about how to present the vast amount of information we've accumulated over the years :-) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Mention kernel-doc Context section
From: Matthew Wilcox <mawil...@microsoft.com> The scripts/kernel-doc processor mentions the ability to add arbitrary section names and suggests including a Context: section. We already have about 450 Context: sections in the kernel, so document it. We also have around 250 Locking: sections in the kernel ... these should probably all be renamed for consistency. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 0268335414ce..c347f5a81cd3 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -116,6 +116,8 @@ Example kernel-doc function comment:: * * Longer description of foobar. * + * Context: Interrupt / locking context of foobar. + * * Return: Description of return value of foobar. */ int foobar(int arg) @@ -220,6 +222,9 @@ The general format of a function and function-like macro kernel-doc comment is:: * * The longer description may have multiple paragraphs. * + * Context: Describes whether the function can sleep, what locks it takes + * or releases, or expects to be held. + * * Return: Describe the return value of foobar. * * The return value description can also have multiple paragraphs, and should @@ -238,6 +243,11 @@ descriptions may span multiple lines. The continuation lines may contain indentation. If a function parameter is ``...`` (varargs), it should be listed in kernel-doc notation as: ``@...:``. +The context in which a function can be called should be documented in a +"Context:" section. Ideally, this will include whether the function +sleeps or can be called from interrupt context, as well as what locks +it takes, releases and expects to be held by its caller. + The return value, if any, should be described in a dedicated section at the end of the comment starting with "Return:". -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Documentation: Fix misconversion of #if
From: Matthew Wilcox <mawil...@microsoft.com> At some stage of the conversion pipeline, something thought that the DocBook entity should be rendered as NUM instead of #. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/kernel-hacking/hacking.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/kernel-hacking/hacking.rst b/Documentation/kernel-hacking/hacking.rst index daf3883b2694..5c4d30e81b7f 100644 --- a/Documentation/kernel-hacking/hacking.rst +++ b/Documentation/kernel-hacking/hacking.rst @@ -690,8 +690,8 @@ not provide the necessary runtime environment and the include files are not tested for it. It is still possible, but not recommended. If you really want to do this, forget about exceptions at least. -NUMif -- +#if +--- It is generally considered cleaner to use macros in header files (or at the top of .c files) to abstract away functions rather than using \`#if' -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 4/5] Fix whitespace in example
From: Matthew Wilcox <mawil...@microsoft.com> Line up the second line in the way that the example purports to be showing. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 466347067f79..85efec8ecc44 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -154,7 +154,7 @@ The ``@argument:`` descriptions may span multiple lines. the previous line, e. g.:: * @argument: some long description - * that continues on next lines + *that continues on next lines or:: -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 3/5] Add scripts/split-man.pl
From: Matthew Wilcox <mawil...@microsoft.com> Instead of asking the user to copy and paste a small perl script from the documentation, just distribute the perl script in the scripts directory. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 30 ++ scripts/split-man.pl | 28 2 files changed, 30 insertions(+), 28 deletions(-) create mode 100755 scripts/split-man.pl diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 64ca081d075e..466347067f79 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -528,32 +528,6 @@ How to use kernel-doc to generate man pages --- If you just want to use kernel-doc to generate man pages you can do this -from the Kernel git tree:: +from the kernel git tree:: - $ scripts/kernel-doc -man $(git grep -l '/\*\*' |grep -v Documentation/) | ./split-man.pl /tmp/man - -Using the small ``split-man.pl`` script below:: - - - #!/usr/bin/perl - - if ($#ARGV < 0) { - die "where do I put the results?\n"; - } - - mkdir $ARGV[0],0777; - $state = 0; - while () { - if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { - if ($state == 1) { close OUT } - $state = 1; - $fn = "$ARGV[0]/$1.9"; - print STDERR "Creating $fn\n"; - open OUT, ">$fn" or die "can't open $fn: $!\n"; - print OUT $_; - } elsif ($state != 0) { - print OUT $_; - } - } - - close OUT; + $ scripts/kernel-doc -man $(git grep -l '/\*\*' -- :^Documentation :^tools) | scripts/split-man.pl /tmp/man diff --git a/scripts/split-man.pl b/scripts/split-man.pl new file mode 100755 index ..bfe16cbe42df --- /dev/null +++ b/scripts/split-man.pl @@ -0,0 +1,28 @@ +#!/usr/bin/perl +# SPDX-License-Identifier: GPL-2.0 +# +# Author: Mauro Carvalho Chehab <mche...@s-opensource.com> +# +# Produce manpages from kernel-doc. +# See Documentation/doc-guide/kernel-doc.rst for instructions + +if ($#ARGV < 0) { + die "where do I put the results?\n"; +} + +mkdir $ARGV[0],0777; +$state = 0; +while () { +if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { + if ($state == 1) { close OUT } + $state = 1; + $fn = "$ARGV[0]/$1.9"; + print STDERR "Creating $fn\n"; + open OUT, ">$fn" or die "can't open $fn: $!\n"; + print OUT $_; +} elsif ($state != 0) { + print OUT $_; +} +} + +close OUT; -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 5/5] Restructure kernel-doc.rst
From: Matthew Wilcox <mawil...@microsoft.com> I found the layout confusing with multiple introductions to what kernel-doc is and how to use it. I made the following changes: - Moved the 'Including kernel-doc comments' section to the end of the document -- we should explain what it *is* before we explain how to integrate it. - Moved the 'Recommendations' subsection to the top. We want people to know what to document before telling them how to do it. - Rewrite the 'Writing kernel-doc comments' section, integrating the 'Recommendations' subsection and a paragraph from 'How to format kernel-doc comments'. - Remove the paragraph about the kernel-doc script; we're supposed to be teaching people how to use punctuation to write pretty documentation, not documenting the build tooling. - Split the 'Parameters and member arguments' section into 'Function parameters' and 'Members'. Structure members are not commonly referred to as arguments. - Integrate the 'private:' and 'public:' tag descriptions into the 'Members' section. - Move the 'In-line member documentation comments' subsection up to be with the 'Members' section. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 476 +++-- 1 file changed, 217 insertions(+), 259 deletions(-) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 85efec8ecc44..2fb7f2bfc245 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -1,202 +1,48 @@ -Including kernel-doc comments -= - -The Linux kernel source files may contain structured documentation comments, or -kernel-doc comments to describe the functions and types and design of the -code. The documentation comments may be included to any of the reStructuredText -documents using a dedicated kernel-doc Sphinx directive extension. - -The kernel-doc directive is of the format:: - - .. kernel-doc:: source - :option: - -The *source* is the path to a source file, relative to the kernel source -tree. The following directive options are supported: - -export: *[source-pattern ...]* - Include documentation for all functions in *source* that have been exported - using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL`` either in *source* or in any - of the files specified by *source-pattern*. - - The *source-pattern* is useful when the kernel-doc comments have been placed - in header files, while ``EXPORT_SYMBOL`` and ``EXPORT_SYMBOL_GPL`` are next to - the function definitions. - - Examples:: - -.. kernel-doc:: lib/bitmap.c - :export: - -.. kernel-doc:: include/net/mac80211.h - :export: net/mac80211/*.c - -internal: *[source-pattern ...]* - Include documentation for all functions and types in *source* that have - **not** been exported using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL`` either - in *source* or in any of the files specified by *source-pattern*. - - Example:: - -.. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c - :internal: - -doc: *title* - Include documentation for the ``DOC:`` paragraph identified by *title* in - *source*. Spaces are allowed in *title*; do not quote the *title*. The *title* - is only used as an identifier for the paragraph, and is not included in the - output. Please make sure to have an appropriate heading in the enclosing - reStructuredText document. - - Example:: - -.. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c - :doc: High Definition Audio over HDMI and Display Port - -functions: *function* *[...]* - Include documentation for each *function* in *source*. - - Example:: - -.. kernel-doc:: lib/bitmap.c - :functions: bitmap_parselist bitmap_parselist_user - -Without options, the kernel-doc directive includes all documentation comments -from the source file. - -The kernel-doc extension is included in the kernel source tree, at -``Documentation/sphinx/kerneldoc.py``. Internally, it uses the -``scripts/kernel-doc`` script to extract the documentation comments from the -source. - -.. _kernel_doc: - Writing kernel-doc comments === -In order to provide embedded, "C" friendly, easy to maintain, but consistent and -extractable overview, function and type documentation, the Linux kernel has -adopted a consistent style for documentation comments. The format for this -documentation is called the kernel-doc format, described below. This style -embeds the documentation within the source files, using a few simple conventions -for adding documentation paragraphs and documenting functions and their -parameters, structures and unions and their members, enumerations, and typedefs. - -.. note:: The kernel-doc format is deceptively similar to gtk-doc or Doxygen, - yet distinctively different, for historical reasons. The kernel source - contains tens of t
[PATCH v2 2/5] Minor fixes to kernel-doc.rst
From: Matthew Wilcox <mawil...@microsoft.com> The author clearly meant to use the word 'which' here. Also replace some tabs with spaces which fixes the syntax highlighting in my editor. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 315b0ecc7a11..64ca081d075e 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -277,7 +277,7 @@ named ``Return``. #) If the descriptive text you provide has lines that begin with some phrase followed by a colon, each of those phrases will be taken - as a new section heading, with probably won't produce the desired + as a new section heading, which probably won't produce the desired effect. Structure, union, and enumeration documentation @@ -324,22 +324,22 @@ It is possible to document nested structs unions, like:: struct { int arg1; int arg2; - } +} struct { void *arg3; int arg4; - } - } - union { + } +} +union { struct { int arg1; int arg2; - } st1; + } st1; struct { void *arg1; int arg2; - } st2; - } bar; + } st2; +} bar; }; .. note:: -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 0/5] Restructure kernel-doc.rst
From: Matthew Wilcox <mawil...@microsoft.com> Jon asked me to redo the Context: patch on top of his current docs tree. Unfortunately, I was on a plane at the time, so I started fixing some other small things, and before I knew it, I'd completely restructured the entire doc-guide/kernel-doc.rst file. v2: Dropped Style Guide section. Added header to split-man.pl as requested by Jon. Credited/blamed Mauro for the perl script, and included a GPL-2 SPDX tag. Matthew Wilcox (5): Add documentation for Context section Minor fixes to kernel-doc.rst Add scripts/split-man.pl Fix whitespace in example Restructure kernel-doc.rst Documentation/doc-guide/kernel-doc.rst | 539 +++-- scripts/split-man.pl | 28 ++ 2 files changed, 276 insertions(+), 291 deletions(-) create mode 100755 scripts/split-man.pl -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 1/5] Add documentation for Context section
From: Matthew Wilcox <mawil...@microsoft.com> This section is mentioned in scripts/kernel-doc, so we should mention it in doc-guide/kernel-doc.rst. There are close to 500 comments using the Context section already, and almost 300 using a Locking section which fulfills much the same purpose (and should probably be converted for consistency). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 25 + 1 file changed, 25 insertions(+) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 722d4525f7cf..315b0ecc7a11 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -117,6 +117,7 @@ Example kernel-doc function comment:: * * Longer description of foobar. * + * Context: Interrupt / locking context of foobar. * Return: Description of return value of foobar. */ int foobar(int argument1, char *argument2) @@ -216,6 +217,9 @@ The general format of a function and function-like macro kernel-doc comment is:: * * The longer description may have multiple paragraphs. * + * Context: Describes whether the function can sleep, what locks it takes, + * releases, or expects to be held. It can extend over multiple + * lines. * Return: Describe the return value of foobar. * * The return value description can also have multiple paragraphs, and should @@ -226,6 +230,24 @@ The brief description following the function name may span multiple lines, and ends with an argument description, a blank comment line, or the end of the comment block. +Function context + + +The context in which a function can be called should be described in a +section named ``Context``. This should include whether the function +sleeps or can be called from interrupt context, as well as what locks +it takes, releases and expects to be held by its caller. + +Examples:: + + * Context: Any context. + * Context: Any context. Takes and releases the RCU lock. + * Context: Any context. Expects to be held by caller. + * Context: Process context. May sleep if @gfp flags permit. + * Context: Process context. Takes and releases . + * Context: Softirq or process context. Takes and releases , BH-safe. + * Context: Interrupt context. + Return values ~ @@ -347,6 +369,9 @@ Typedefs with function prototypes can also be documented:: * @arg2: description of arg2 * * Description of the type. + * + * Context: Locking context. + * Return: Meaning of the return value. */ typedef void (*type_name)(struct v4l2_ctrl *arg1, void *arg2); -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/6] Restructure kernel-doc.rst
On Tue, Feb 13, 2018 at 01:03:18PM -0800, Randy Dunlap wrote: > On 02/13/2018 12:59 PM, Jonathan Corbet wrote: > > I think this makes sense to do, but I really would like to see an intro > > comment on the split-man.pl script itself. Somebody wandering through > > the scripts directory should be able to look at the top of the file and > > know what the script does without having to reverse-engineer delightful > > stuff like: > > > >> if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { Fixed in v2 ;-) > >> Fix whitespace in example > >> Restructure kernel-doc.rst > >> Add a Style Guide section > > > > This last one worries me a bit because we're starting to impinge a bit on > > the coding style document. If we're going to give guidance on things > > like parameter names, I think that coding-style.rst is the right place > > for it. Dropped for v2. > As long as the examples don't dictate style... > E.g., Matthew likes to use: > > + * @gfp: Memory allocation flags. > > a. Capitalize the first word after @param: > b. end with a period (even though it's not a sentence) > > I disagree with both of those. OK, so which do you think looks better: https://www.kernel.org/doc/html/latest/core-api/idr.html#c.idr_get_cursor https://www.kernel.org/doc/html/latest/core-api/idr.html#c.idr_alloc_u32 (yes, I have failed to be consistent. I shall apply appropriate amounts of self-flagellation and fix them shortly.) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/6] Add documentation for Context section
From: Matthew Wilcox <mawil...@microsoft.com> This section is mentioned in scripts/kernel-doc, so we should mention it in doc-guide/kernel-doc.rst. There are close to 500 comments using the Context section already, and almost 300 using a Locking section which fulfills much the same purpose (and should probably be converted for consistency). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 25 + 1 file changed, 25 insertions(+) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 722d4525f7cf..315b0ecc7a11 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -117,6 +117,7 @@ Example kernel-doc function comment:: * * Longer description of foobar. * + * Context: Interrupt / locking context of foobar. * Return: Description of return value of foobar. */ int foobar(int argument1, char *argument2) @@ -216,6 +217,9 @@ The general format of a function and function-like macro kernel-doc comment is:: * * The longer description may have multiple paragraphs. * + * Context: Describes whether the function can sleep, what locks it takes, + * releases, or expects to be held. It can extend over multiple + * lines. * Return: Describe the return value of foobar. * * The return value description can also have multiple paragraphs, and should @@ -226,6 +230,24 @@ The brief description following the function name may span multiple lines, and ends with an argument description, a blank comment line, or the end of the comment block. +Function context + + +The context in which a function can be called should be described in a +section named ``Context``. This should include whether the function +sleeps or can be called from interrupt context, as well as what locks +it takes, releases and expects to be held by its caller. + +Examples:: + + * Context: Any context. + * Context: Any context. Takes and releases the RCU lock. + * Context: Any context. Expects to be held by caller. + * Context: Process context. May sleep if @gfp flags permit. + * Context: Process context. Takes and releases . + * Context: Softirq or process context. Takes and releases , BH-safe. + * Context: Interrupt context. + Return values ~ @@ -347,6 +369,9 @@ Typedefs with function prototypes can also be documented:: * @arg2: description of arg2 * * Description of the type. + * + * Context: Locking context. + * Return: Meaning of the return value. */ typedef void (*type_name)(struct v4l2_ctrl *arg1, void *arg2); -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 5/6] Restructure kernel-doc.rst
From: Matthew Wilcox <mawil...@microsoft.com> I found the layout confusing with multiple introductions to what kernel-doc is and how to use it. I made the following changes: - Moved the 'Including kernel-doc comments' section to the end of the document -- we should explain what it *is* before we explain how to integrate it. - Moved the 'Recommendations' subsection to the top. We want people to know what to document before telling them how to do it. - Rewrite the 'Writing kernel-doc comments' section, integrating the 'Recommendations' subsection and a paragraph from 'How to format kernel-doc comments'. - Remove the paragraph about the kernel-doc script; we're supposed to be teaching people how to use punctuation to write pretty documentation, not documenting the build tooling. - Split the 'Parameters and member arguments' section into 'Function parameters' and 'Members'. Structure members are not commonly referred to as arguments. - Integrate the 'private:' and 'public:' tag descriptions into the 'Members' section. - Move the 'In-line member documentation comments' subsection up to be with the 'Members' section. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 476 +++-- 1 file changed, 217 insertions(+), 259 deletions(-) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 85efec8ecc44..2fb7f2bfc245 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -1,202 +1,48 @@ -Including kernel-doc comments -= - -The Linux kernel source files may contain structured documentation comments, or -kernel-doc comments to describe the functions and types and design of the -code. The documentation comments may be included to any of the reStructuredText -documents using a dedicated kernel-doc Sphinx directive extension. - -The kernel-doc directive is of the format:: - - .. kernel-doc:: source - :option: - -The *source* is the path to a source file, relative to the kernel source -tree. The following directive options are supported: - -export: *[source-pattern ...]* - Include documentation for all functions in *source* that have been exported - using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL`` either in *source* or in any - of the files specified by *source-pattern*. - - The *source-pattern* is useful when the kernel-doc comments have been placed - in header files, while ``EXPORT_SYMBOL`` and ``EXPORT_SYMBOL_GPL`` are next to - the function definitions. - - Examples:: - -.. kernel-doc:: lib/bitmap.c - :export: - -.. kernel-doc:: include/net/mac80211.h - :export: net/mac80211/*.c - -internal: *[source-pattern ...]* - Include documentation for all functions and types in *source* that have - **not** been exported using ``EXPORT_SYMBOL`` or ``EXPORT_SYMBOL_GPL`` either - in *source* or in any of the files specified by *source-pattern*. - - Example:: - -.. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c - :internal: - -doc: *title* - Include documentation for the ``DOC:`` paragraph identified by *title* in - *source*. Spaces are allowed in *title*; do not quote the *title*. The *title* - is only used as an identifier for the paragraph, and is not included in the - output. Please make sure to have an appropriate heading in the enclosing - reStructuredText document. - - Example:: - -.. kernel-doc:: drivers/gpu/drm/i915/intel_audio.c - :doc: High Definition Audio over HDMI and Display Port - -functions: *function* *[...]* - Include documentation for each *function* in *source*. - - Example:: - -.. kernel-doc:: lib/bitmap.c - :functions: bitmap_parselist bitmap_parselist_user - -Without options, the kernel-doc directive includes all documentation comments -from the source file. - -The kernel-doc extension is included in the kernel source tree, at -``Documentation/sphinx/kerneldoc.py``. Internally, it uses the -``scripts/kernel-doc`` script to extract the documentation comments from the -source. - -.. _kernel_doc: - Writing kernel-doc comments === -In order to provide embedded, "C" friendly, easy to maintain, but consistent and -extractable overview, function and type documentation, the Linux kernel has -adopted a consistent style for documentation comments. The format for this -documentation is called the kernel-doc format, described below. This style -embeds the documentation within the source files, using a few simple conventions -for adding documentation paragraphs and documenting functions and their -parameters, structures and unions and their members, enumerations, and typedefs. - -.. note:: The kernel-doc format is deceptively similar to gtk-doc or Doxygen, - yet distinctively different, for historical reasons. The kernel source - contains tens of t
[PATCH 3/6] Add scripts/split-man.pl
From: Matthew Wilcox <mawil...@microsoft.com> Instead of asking the user to copy and paste a small perl script from the documentation, just distribute the perl script in the scripts directory. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 30 ++ scripts/split-man.pl | 22 ++ 2 files changed, 24 insertions(+), 28 deletions(-) create mode 100755 scripts/split-man.pl diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 64ca081d075e..466347067f79 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -528,32 +528,6 @@ How to use kernel-doc to generate man pages --- If you just want to use kernel-doc to generate man pages you can do this -from the Kernel git tree:: +from the kernel git tree:: - $ scripts/kernel-doc -man $(git grep -l '/\*\*' |grep -v Documentation/) | ./split-man.pl /tmp/man - -Using the small ``split-man.pl`` script below:: - - - #!/usr/bin/perl - - if ($#ARGV < 0) { - die "where do I put the results?\n"; - } - - mkdir $ARGV[0],0777; - $state = 0; - while () { - if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { - if ($state == 1) { close OUT } - $state = 1; - $fn = "$ARGV[0]/$1.9"; - print STDERR "Creating $fn\n"; - open OUT, ">$fn" or die "can't open $fn: $!\n"; - print OUT $_; - } elsif ($state != 0) { - print OUT $_; - } - } - - close OUT; + $ scripts/kernel-doc -man $(git grep -l '/\*\*' -- :^Documentation :^tools) | scripts/split-man.pl /tmp/man diff --git a/scripts/split-man.pl b/scripts/split-man.pl new file mode 100755 index ..bb39b12f691d --- /dev/null +++ b/scripts/split-man.pl @@ -0,0 +1,22 @@ +#!/usr/bin/perl + +if ($#ARGV < 0) { + die "where do I put the results?\n"; +} + +mkdir $ARGV[0],0777; +$state = 0; +while () { +if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { + if ($state == 1) { close OUT } + $state = 1; + $fn = "$ARGV[0]/$1.9"; + print STDERR "Creating $fn\n"; + open OUT, ">$fn" or die "can't open $fn: $!\n"; + print OUT $_; +} elsif ($state != 0) { + print OUT $_; +} +} + +close OUT; -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 6/6] Add a Style Guide section
From: Matthew Wilcox <mawil...@microsoft.com> Let's try to be consistent and copy each other's best practices. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 42 ++ 1 file changed, 42 insertions(+) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 2fb7f2bfc245..0253e51b76cf 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -411,6 +411,48 @@ The title following ``DOC:`` acts as a heading within the source file, but also as an identifier for extracting the documentation comment. Thus, the title must be unique within the file. +Style guide +--- + +It is helpful to maintain a consistent style across the kernel. It helps +the reader who can rely on conventions to understand a part of the kernel +which is new to them. It also helps the writer who can simply copy and +paste function parameter descriptions instead of coming up with new and +excitingly different ways to say the same thing. + +Context names +~ + +Refer to process context, softirq context and hardirq context. Interrupt +context refers to both hard and soft interrupts. Bottom half context +is deprecated; it is a synonym for softirq context. Tasklet context +should not be used; tasklets run in softirq context. Do not use user +context; this is a synonym for process context. + +Common function parameters +~~ + +If you pass ``GFP_`` flags to a function, name the parameter ``gfp`` and +document it like this:: + + * @gfp: Memory allocation flags. + +If your function takes a ``struct file *`` argument, name the parameter ``file`` +(unless you have a good reason to use another name) and document it like +this:: + + * @file: File pointer. + +If your function takes a ``struct dentry *`` argument, name the +parameter ``dentry`` and document it like this:: + + * @dentry: Directory Entry pointer. + +If your function takes a ``struct inode *`` argument, name the parameter +``inode`` and document it like this:: + + * @inode: Inode pointer. + Including kernel-doc comments = -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/6] Minor fixes to kernel-doc.rst
From: Matthew Wilcox <mawil...@microsoft.com> The author clearly meant to use the word 'which' here. Also replace some tabs with spaces which fixes the syntax highlighting in my editor. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> --- Documentation/doc-guide/kernel-doc.rst | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst index 315b0ecc7a11..64ca081d075e 100644 --- a/Documentation/doc-guide/kernel-doc.rst +++ b/Documentation/doc-guide/kernel-doc.rst @@ -277,7 +277,7 @@ named ``Return``. #) If the descriptive text you provide has lines that begin with some phrase followed by a colon, each of those phrases will be taken - as a new section heading, with probably won't produce the desired + as a new section heading, which probably won't produce the desired effect. Structure, union, and enumeration documentation @@ -324,22 +324,22 @@ It is possible to document nested structs unions, like:: struct { int arg1; int arg2; - } +} struct { void *arg3; int arg4; - } - } - union { + } +} +union { struct { int arg1; int arg2; - } st1; + } st1; struct { void *arg1; int arg2; - } st2; - } bar; + } st2; +} bar; }; .. note:: -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/6] Restructure kernel-doc.rst
From: Matthew Wilcox <mawil...@microsoft.com> Jon asked me to redo the Context: patch on top of his current docs tree. Unfortunately, I was on a plane at the time, so I started fixing some other small things, and before I knew it, I'd completely restructured the entire doc-guide/kernel-doc.rst file. Feel free to not take it all, particularly the last 'Style Guide' patch. Matthew Wilcox (6): Add documentation for Context section Minor fixes to kernel-doc.rst Add scripts/split-man.pl Fix whitespace in example Restructure kernel-doc.rst Add a Style Guide section Documentation/doc-guide/kernel-doc.rst | 581 - scripts/split-man.pl | 22 ++ 2 files changed, 312 insertions(+), 291 deletions(-) create mode 100755 scripts/split-man.pl -- 2.15.1 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] mm, page_alloc: extend kernelcore and movablecore for percent
On Thu, Feb 15, 2018 at 03:45:25PM +0100, Michal Hocko wrote: > > When the amount of kernel > > memory is well bounded for certain systems, it is better to aggressively > > reclaim from existing MIGRATE_UNMOVABLE pageblocks rather than eagerly > > fallback to others. > > > > We have additional patches that help with this fragmentation if you're > > interested, specifically kcompactd compaction of MIGRATE_UNMOVABLE > > pageblocks triggered by fallback of non-__GFP_MOVABLE allocations and > > draining of pcp lists back to the zone free area to prevent stranding. > > Yes, I think we need a proper fix. (Ab)using zone_movable for this > usecase is just sad. What if ... on startup, slab allocated a MAX_ORDER page for itself. It would then satisfy its own page allocation requests from this giant page. If we start to run low on memory in the rest of the system, slab can be induced to return some of it via its shrinker. If slab runs low on memory, it tries to allocate another MAX_ORDER page for itself. I think even this should reduce fragmentation. We could enhance the fragmentation reduction by noticing when somebody else releases a page that was previously part of a slab MAX_ORDER page and handing that page back to slab. When slab notices that it has an entire MAX_ORDER page free (and sufficient other memory on hand that it's unlikely to need it soon), it can hand that MAX_ORDER page back to the page allocator. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] linux-next: SLIMbus: doc: Fix a warning "Title underline too short"
On Mon, Feb 19, 2018 at 10:55:50PM +0900, Masanari Iida wrote: > Driver and Controller APIs: > --- > +--- > .. kernel-doc:: include/linux/slimbus.h Is this the right fix? Shouldn't we rather delete the : instead? We don't normally have a colon at the end of subsection titles. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] doc-guide: kernel-doc: add comment about formatting verification
On Tue, Feb 20, 2018 at 04:54:44PM +0200, Mike Rapoport wrote: > level. > > +Running the ``kernel-doc`` tool with increased verbosity and without actual > +output generation may be used to verify proper formating of the "formatting" > +documentation comments. For example:: > + > + scripts/kernel-doc -v -none drivers/foo/bar.c > + > Function documentation > -- And that's not exactly true. "make W=1" will run that exact command. For example: $ make W=1 net/ipv4/ (...) CC net/ipv4/tcp_input.o net/ipv4/tcp_input.c:4279: warning: Excess function parameter 'dest' description in 'tcp_try_coalesce' CC net/ipv4/tcp_output.o net/ipv4/tcp_output.c:3177: warning: Function parameter or member 'sk' not described in 'tcp_make_synack' (...) perhaps sneak that information in there somewhere. Also it'll prompt people to run with W=1, so perhaps they'll fix their more-easily-fixed warnings ;-) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] mm, page_alloc: extend kernelcore and movablecore for percent
On Fri, Feb 16, 2018 at 09:44:25AM -0600, Christopher Lameter wrote: > On Thu, 15 Feb 2018, Matthew Wilcox wrote: > > What I was proposing was an intermediate page allocator where slab would > > request 2MB for its own uses all at once, then allocate pages from that to > > individual slabs, so allocating a kmalloc-32 object and a dentry object > > would result in 510 pages of memory still being available for any slab > > that needed it. > > Well thats not really going to work since you would be mixing objects of > different sizes which may present more fragmentation problems within the > 2M later if they are freed and more objects are allocated. I don't understand this response. I'm not suggesting mixing objects of different sizes within the same page. The vast majority of slabs use order-0 pages, a few use order-1 pages and larger sizes are almost unheard of. I'm suggesting the slab have it's own private arena of pages that it uses for allocating pages to slabs; when an entire page comes free in a slab, it is returned to the arena. When the arena is empty, slab requests another arena from the page allocator. If you're concerned about order-0 allocations fragmenting the arena for order-1 slabs, then we could have separate arenas for order-0 and order-1. But there should be no more fragmentation caused by sticking within an arena for page allocations than there would be by spreading slab allocations across all memory. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] mm, page_alloc: extend kernelcore and movablecore for percent
On Thu, Feb 15, 2018 at 09:49:00AM -0600, Christopher Lameter wrote: > On Thu, 15 Feb 2018, Matthew Wilcox wrote: > > > What if ... on startup, slab allocated a MAX_ORDER page for itself. > > It would then satisfy its own page allocation requests from this giant > > page. If we start to run low on memory in the rest of the system, slab > > can be induced to return some of it via its shrinker. If slab runs low > > on memory, it tries to allocate another MAX_ORDER page for itself. > > The inducing of releasing memory back is not there but you can run SLUB > with MAX_ORDER allocations by passing "slab_min_order=9" or so on bootup. Maybe we should try this patch in order to automatically scale the slub page size with the amount of memory in the machine? diff --git a/mm/internal.h b/mm/internal.h index e6bd35182dae..7059a8389194 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -167,6 +167,7 @@ extern void prep_compound_page(struct page *page, unsigned int order); extern void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags); extern int user_min_free_kbytes; +extern unsigned long __meminitdata nr_kernel_pages; #if defined CONFIG_COMPACTION || defined CONFIG_CMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ef9c259db041..3c51bb22403f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -264,7 +264,7 @@ int min_free_kbytes = 1024; int user_min_free_kbytes = -1; int watermark_scale_factor = 10; -static unsigned long __meminitdata nr_kernel_pages; +unsigned long __meminitdata nr_kernel_pages; static unsigned long __meminitdata nr_all_pages; static unsigned long __meminitdata dma_reserve; diff --git a/mm/slub.c b/mm/slub.c index e381728a3751..abca4a6e9b6c 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4194,6 +4194,23 @@ void __init kmem_cache_init(void) if (debug_guardpage_minorder()) slub_max_order = 0; + if (slub_min_order == 0) { + unsigned long numentries = nr_kernel_pages; + + /* +* Above 4GB, we start to care more about fragmenting large +* pages than about using the minimum amount of memory. +* Scale the slub page size at half the rate that we scale +* the memory size; at 4GB we double the page size to 8k, +* 16GB to 16k, 64GB to 32k, 256GB to 64k. +*/ + while (numentries > (4UL << 30)) { + if (slub_min_order >= slub_max_order) + break; + slub_min_order++; + numentries /= 4; + } + } kmem_cache_node = _kmem_cache_node; kmem_cache = _kmem_cache; -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] mm, page_alloc: extend kernelcore and movablecore for percent
On Thu, Feb 15, 2018 at 09:49:00AM -0600, Christopher Lameter wrote: > On Thu, 15 Feb 2018, Matthew Wilcox wrote: > > What if ... on startup, slab allocated a MAX_ORDER page for itself. > > It would then satisfy its own page allocation requests from this giant > > page. If we start to run low on memory in the rest of the system, slab > > can be induced to return some of it via its shrinker. If slab runs low > > on memory, it tries to allocate another MAX_ORDER page for itself. > > The inducing of releasing memory back is not there but you can run SLUB > with MAX_ORDER allocations by passing "slab_min_order=9" or so on bootup. This is subtly different from the idea that I had. If you set slub_min_order to 9, then slub will allocate 2MB pages for each slab, so allocating one object from kmalloc-32 and one object from dentry will cause 4MB to be taken from the system. What I was proposing was an intermediate page allocator where slab would request 2MB for its own uses all at once, then allocate pages from that to individual slabs, so allocating a kmalloc-32 object and a dentry object would result in 510 pages of memory still being available for any slab that needed it. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch 1/2] mm, page_alloc: extend kernelcore and movablecore for percent
On Fri, Feb 16, 2018 at 10:08:28AM -0600, Christopher Lameter wrote: > On Fri, 16 Feb 2018, Matthew Wilcox wrote: > > I don't understand this response. I'm not suggesting mixing objects > > of different sizes within the same page. The vast majority of slabs > > use order-0 pages, a few use order-1 pages and larger sizes are almost > > unheard of. I'm suggesting the slab have it's own private arena of pages > > that it uses for allocating pages to slabs; when an entire page comes > > free in a slab, it is returned to the arena. When the arena is empty, > > slab requests another arena from the page allocator. > > This just shifts the fragmentation problem because the 2M page cannot be > released until all 4k or 8k pages within that 2M page are freed. How is > that different from the page allocator which cannot coalesce an 2M page > until all fragments have been released? I'm not proposing releasing this 2MB page, unless it naturally frees up. I'm saying that by restricting allocations to be within this 2MB page, we prevent allocating from the adjacent 2MB page. The workload I'm thinking of looks like this ... maybe the result of running 'file' on every inode in a directory: do { Allocate an inode Allocate a page of pagecache } while (lots of times); naively, we allocate a page for the inode slab, then 3-6 pages for page cache (depending on the filesystem), then we allocate another page for the inode slab, then another 3-6 pages of page cache, and so on. So the pages end up looking like this: IPIP|IPPP|PPIP|IPIP|... Now we need an order-3 allocation. We can't get there just by releasing page cache pages because there's inode slab pages in there, so we need to shrink the inode caches as well. I'm proposing: II00||||PP... and we can get our order-3 allocation just by releasing page cache pages. > The kernelcore already does something similar by limiting the > general unmovable allocs to a section of memory. Right! But Michal's unhappy about kernelcore (see the beginning of this thread), and so I'm proposing an alternative. > Maybe what we should do is raise the lowest allocation size instead and > allocate 2^x groups of pages to certain purposes? > > I.e. have a base allocation size of 16k and if the alloc was a page cache > page then use the remainder for the neigboring pages. Yes, there are a lot of ideas like this floating around; I know Kirill's interested in this kind of thing not just for THP but also for faultaround. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2] errseq: Add to documentation tree
From: Matthew Wilcox <mawil...@microsoft.com> - Move errseq.rst into core-api - Add errseq to the core-api index - Promote the header to a more prominent header type, otherwise we get three entries in the table of contents. - Reformat the table to look nicer and be a little more proportional in terms of horizontal width per bit (the SF bit is still disproportionately large, but there's no way to fix that). - Include errseq kernel-doc in the errseq.rst - Neaten some kernel-doc markup Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> Reviewed-by: Jeff Layton <jlay...@redhat.com> --- Documentation/{ => core-api}/errseq.rst | 20 +- Documentation/core-api/index.rst| 1 + include/linux/errseq.h | 2 +- lib/errseq.c| 37 +++-- 4 files changed, 38 insertions(+), 22 deletions(-) rename Documentation/{ => core-api}/errseq.rst (92%) diff --git a/Documentation/errseq.rst b/Documentation/core-api/errseq.rst similarity index 92% rename from Documentation/errseq.rst rename to Documentation/core-api/errseq.rst index 4c29bd5afbc5..ff332e272405 100644 --- a/Documentation/errseq.rst +++ b/Documentation/core-api/errseq.rst @@ -1,5 +1,7 @@ += The errseq_t datatype = + An errseq_t is a way of recording errors in one place, and allowing any number of "subscribers" to tell whether it has changed since a previous point where it was sampled. @@ -21,12 +23,13 @@ a flag to tell whether the value has been sampled since a new value was recorded. That allows us to avoid bumping the counter if no one has sampled it since the last time an error was recorded. -Thus we end up with a value that looks something like this:: +Thus we end up with a value that looks something like this: -bit: 31..131211..0 -+-+++ -| counter | SF | errno | -+-+++ ++--+++ +| 31..13 | 12 | 11..0 | ++--+++ +| counter | SF | errno | ++--+++ The general idea is for "watchers" to sample an errseq_t value and keep it as a running cursor. That value can later be used to tell whether @@ -42,6 +45,7 @@ has ever been an error set since it was first initialized. API usage = + Let me tell you a story about a worker drone. Now, he's a good worker overall, but the company is a little...management heavy. He has to report to 77 supervisors today, and tomorrow the "big boss" is coming in @@ -125,6 +129,7 @@ not usable by anyone else. Serializing errseq_t cursor updates === + Note that the errseq_t API does not protect the errseq_t cursor during a check_and_advance_operation. Only the canonical error code is handled atomically. In a situation where more than one task might be using the @@ -147,3 +152,8 @@ errseq_check_and_advance after taking the lock. e.g.:: That avoids the spinlock in the common case where nothing has changed since the last time it was checked. + +Functions += + +.. kernel-doc:: lib/errseq.c diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index eb16ba30aeb6..b226d05df1c3 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -22,6 +22,7 @@ Core utilities flexible-arrays librs genalloc + errseq Interfaces for kernel debugging === diff --git a/include/linux/errseq.h b/include/linux/errseq.h index 6ffae9c5052d..fc270768 100644 --- a/include/linux/errseq.h +++ b/include/linux/errseq.h @@ -1,6 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * See Documentation/errseq.rst and lib/errseq.c + * See Documentation/core-api/errseq.rst and lib/errseq.c */ #ifndef _LINUX_ERRSEQ_H #define _LINUX_ERRSEQ_H diff --git a/lib/errseq.c b/lib/errseq.c index 79cc66897db4..df782418b333 100644 --- a/lib/errseq.c +++ b/lib/errseq.c @@ -46,14 +46,14 @@ * @eseq: errseq_t field that should be set * @err: error to set (must be between -1 and -MAX_ERRNO) * - * This function sets the error in *eseq, and increments the sequence counter + * This function sets the error in @eseq, and increments the sequence counter * if the last sequence was sampled at some point in the past. * * Any error set will always overwrite an existing error. * - * We do return the latest value here, primarily for debugging purposes. The - * return value should not be used as a previously sampled value in later calls - * as it will not have the SEEN flag set. + * Return: The previous v
Re: [PATCH] Add errseq_t documentation to the tree
On Fri, Dec 22, 2017 at 08:29:34AM -0500, Jeff Layton wrote: > > +++ b/Documentation/core-api/index.rst > > @@ -22,6 +22,7 @@ Core utilities > > flexible-arrays > > librs > > genalloc > > + ../errseq > > > > Should we also move the file into core-api/ dir? I was wondering the same thing. I'll do a v2. I was also thinking about including the kernel-doc from lib/errseq.c into this file. Seems like the logical thing to do. > Reviewed-by: Jeff LaytonMerci! -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Add errseq_t documentation to the tree
- Add it under 'Core API' because I think that's where it lives. - Promote the header to a more prominent header type, otherwise we get three entries in the table of contents. - Reformat the table to look nicer and be a little more proportional in terms of horizontal width per bit (the SF bit is still disproportionately large, but there's no way to fix that). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index eb16ba30aeb6..b8ec120c24f9 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -22,6 +22,7 @@ Core utilities flexible-arrays librs genalloc + ../errseq Interfaces for kernel debugging === diff --git a/Documentation/errseq.rst b/Documentation/errseq.rst index 4c29bd5afbc5..7c3ac9639ebf 100644 --- a/Documentation/errseq.rst +++ b/Documentation/errseq.rst @@ -1,5 +1,7 @@ += The errseq_t datatype = + An errseq_t is a way of recording errors in one place, and allowing any number of "subscribers" to tell whether it has changed since a previous point where it was sampled. @@ -21,12 +23,13 @@ a flag to tell whether the value has been sampled since a new value was recorded. That allows us to avoid bumping the counter if no one has sampled it since the last time an error was recorded. -Thus we end up with a value that looks something like this:: +Thus we end up with a value that looks something like this: -bit: 31..131211..0 -+-+++ -| counter | SF | errno | -+-+++ ++--+++ +| 31..13 | 12 | 11..0 | ++--+++ +| counter | SF | errno | ++--+++ The general idea is for "watchers" to sample an errseq_t value and keep it as a running cursor. That value can later be used to tell whether @@ -42,6 +45,7 @@ has ever been an error set since it was first initialized. API usage = + Let me tell you a story about a worker drone. Now, he's a good worker overall, but the company is a little...management heavy. He has to report to 77 supervisors today, and tomorrow the "big boss" is coming in @@ -125,6 +129,7 @@ not usable by anyone else. Serializing errseq_t cursor updates === + Note that the errseq_t API does not protect the errseq_t cursor during a check_and_advance_operation. Only the canonical error code is handled atomically. In a situation where more than one task might be using the -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Documentation: security/credentials.rst: explain need to sort group_list
On Wed, Jan 03, 2018 at 08:01:15AM +1100, NeilBrown wrote: > > +When replacing the group list, the new list must be sorted before it > +is added to the credential, as a binary search is used to test for > +membership. In practice, this means ``groups_sort()`` should be For a .rst file, shouldn't we be using :c:func:`groups_sort` instead of ``groups_sort()``? > +called before ``set_groups()`` or ``set_current_groups()``. > +``groups_sort()`` must not be called on a ``struct group_list`` which > +is shared as it may permute elements as part of the sorting process > +even if the array is already sorted. > > When the credential set is ready, it should be committed to the current > process > by calling:: > -- > 2.14.0.rc0.dirty > -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: rfc: remove print_vma_addr ? (was Re: [PATCH 00/16] remove eight obsolete architectures)
On Thu, Mar 15, 2018 at 09:56:46AM -0700, Joe Perches wrote: > I have a patchset that creates a vsprintf extension for > print_vma_addr and removes all the uses similar to the > print_symbol() removal. > > This now avoids any possible printk interleaving. > > Unfortunately, without some #ifdef in vsprintf, which > I would like to avoid, it increases the nommu kernel > size by ~500 bytes. > > Anyone think this is acceptable? > > Here's the overall patch, but I have it as a series > --- > Documentation/core-api/printk-formats.rst | 9 + > arch/arm64/kernel/traps.c | 13 +++ > arch/mips/mm/fault.c | 16 - > arch/parisc/mm/fault.c| 15 > arch/riscv/kernel/traps.c | 11 +++--- > arch/s390/mm/fault.c | 7 ++-- > arch/sparc/mm/fault_32.c | 8 ++--- > arch/sparc/mm/fault_64.c | 8 ++--- > arch/tile/kernel/signal.c | 9 ++--- > arch/um/kernel/trap.c | 13 +++ > arch/x86/kernel/signal.c | 10 ++ > arch/x86/kernel/traps.c | 18 -- > arch/x86/mm/fault.c | 12 +++ > include/linux/mm.h| 1 - > lib/vsprintf.c| 58 > ++- > mm/memory.c | 33 -- > 16 files changed, 112 insertions(+), 129 deletions(-) This doesn't feel like a huge win since it's only called ~once per architecture. I'd be more excited if it made the printing of the whole thing standardised; eg we have a print_fault() function in mm/memory.c which takes a suitable set of arguments. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Improve mutex documentation
On Wed, Mar 14, 2018 at 01:56:31PM -0700, Andrew Morton wrote: > My memory is weak and our documentation is awful. What does > mutex_lock_killable() actually do and how does it differ from > mutex_lock_interruptible()? From: Matthew Wilcox <mawil...@microsoft.com> Add kernel-doc for mutex_lock_killable() and mutex_lock_io(). Reword the kernel-doc for mutex_lock_interruptible(). Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 858a07590e39..2048359f33d2 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -1082,15 +1082,16 @@ static noinline int __sched __mutex_lock_interruptible_slowpath(struct mutex *lock); /** - * mutex_lock_interruptible - acquire the mutex, interruptible - * @lock: the mutex to be acquired + * mutex_lock_interruptible() - Acquire the mutex, interruptible by signals. + * @lock: The mutex to be acquired. * - * Lock the mutex like mutex_lock(), and return 0 if the mutex has - * been acquired or sleep until the mutex becomes available. If a - * signal arrives while waiting for the lock then this function - * returns -EINTR. + * Lock the mutex like mutex_lock(). If a signal is delivered while the + * process is sleeping, this function will return without acquiring the + * mutex. * - * This function is similar to (but not equivalent to) down_interruptible(). + * Context: Process context. + * Return: 0 if the lock was successfully acquired or %-EINTR if a + * signal arrived. */ int __sched mutex_lock_interruptible(struct mutex *lock) { @@ -1104,6 +1105,18 @@ int __sched mutex_lock_interruptible(struct mutex *lock) EXPORT_SYMBOL(mutex_lock_interruptible); +/** + * mutex_lock_killable() - Acquire the mutex, interruptible by fatal signals. + * @lock: The mutex to be acquired. + * + * Lock the mutex like mutex_lock(). If a signal which will be fatal to + * the current process is delivered while the process is sleeping, this + * function will return without acquiring the mutex. + * + * Context: Process context. + * Return: 0 if the lock was successfully acquired or %-EINTR if a + * fatal signal arrived. + */ int __sched mutex_lock_killable(struct mutex *lock) { might_sleep(); @@ -1115,6 +1128,16 @@ int __sched mutex_lock_killable(struct mutex *lock) } EXPORT_SYMBOL(mutex_lock_killable); +/** + * mutex_lock_io() - Acquire the mutex and mark the process as waiting for I/O + * @lock: The mutex to be acquired. + * + * Lock the mutex like mutex_lock(). While the task is waiting for this + * mutex, it will be accounted as being in the IO wait state by the + * scheduler. + * + * Context: Process context. + */ void __sched mutex_lock_io(struct mutex *lock) { int token; -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] kernel-doc: Remove __sched markings
From: Matthew Wilcox <mawil...@microsoft.com> I find the __sched annotations unaesthetic in the kernel-doc. Remove them like we remove __inline, __weak, __init and so on. Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> diff --git a/scripts/kernel-doc b/scripts/kernel-doc index ae3cac118a11..eb986a7809d3 100755 --- a/scripts/kernel-doc +++ b/scripts/kernel-doc @@ -1578,6 +1578,7 @@ sub dump_function($$) { $prototype =~ s/__meminit +//; $prototype =~ s/__must_check +//; $prototype =~ s/__weak +//; +$prototype =~ s/__sched +//; my $define = $prototype =~ s/^#\s*define\s+//; #ak added $prototype =~ s/__attribute__\s*\(\( (?: -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Improve mutex documentation
On Thu, Mar 15, 2018 at 03:12:30PM +0300, Kirill Tkhai wrote: > > +/** > > + * mutex_lock_killable() - Acquire the mutex, interruptible by fatal > > signals. > > Shouldn't we clarify that fatal signals are SIGKILL only? It's more complicated than it might seem (... welcome to signal handling!) If you send SIGINT to a task that's waiting on a mutex_killable(), it will still die. I *think* that's due to the code in complete_signal(): if (sig_fatal(p, sig) && !(signal->flags & SIGNAL_GROUP_EXIT) && !sigismember(>real_blocked, sig) && (sig == SIGKILL || !p->ptrace)) { ... sigaddset(>pending.signal, SIGKILL); You're correct that this code only checks for SIGKILL, but any fatal signal will result in the signal group receiving SIGKILL. Unless I've misunderstood, and it wouldn't be the first time I've misunderstood signal handling. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH net-next 00/12] fscache: Fixes, traces and development
On Fri, Apr 06, 2018 at 11:32:11AM -0700, Linus Torvalds wrote: > On Fri, Apr 6, 2018 at 11:21 AM, Linus Torvalds >wrote: > > > > No, but if you can redo the pull request part so that the diffstat I > > get will match the diffstat I see in the pull request, that would be > > good. > > Oh, and can you please make sure there is a "[GIT PULL]" in the > subject line for your pull requests? > > Particularly during the merge window (not so much later) I end up > having a separate filtered list of emails that I look at that mention > "git pull". > > Your pull requests don't seem to match that, so your pull requests > don't actually even show up in my list of pending pull requests. We have out of date information in Documentation ... Jon, please consider applying: diff --git a/Documentation/process/submitting-patches.rst b/Documentation/process/submitting-patches.rst index f7152ed565e5..908bb55be407 100644 --- a/Documentation/process/submitting-patches.rst +++ b/Documentation/process/submitting-patches.rst @@ -761,7 +761,7 @@ requests, especially from new, unknown developers. If in doubt you can use the pull request as the cover letter for a normal posting of the patch series, giving the maintainer the option of using either. -A pull request should have [GIT] or [PULL] in the subject line. The +A pull request should have [GIT PULL] in the subject line. The request itself should include the repository name and the branch of interest on a single line; it should look something like:: -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 1/2] mm: introduce ARCH_HAS_PTE_SPECIAL
On Tue, Apr 10, 2018 at 05:25:50PM +0200, Laurent Dufour wrote: > arch/powerpc/include/asm/pte-common.h | 3 --- > arch/riscv/Kconfig | 1 + > arch/s390/Kconfig | 1 + You forgot to delete __HAVE_ARCH_PTE_SPECIAL from arch/riscv/include/asm/pgtable-bits.h -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/32] docs/vm: convert to ReST format
On Fri, Apr 13, 2018 at 01:55:51PM -0600, Jonathan Corbet wrote: > > I believe that keeping the mm docs together will give better visibility of > > what (little) mm documentation we have and will make the updates easier. > > The documents that fit well into a certain topic could be linked there. For > > instance: > > ...but this sounds like just the opposite...? > > I've had this conversation with folks in a number of subsystems. > Everybody wants to keep their documentation together in one place - it's > easier for the developers after all. But for the readers I think it's > objectively worse. It perpetuates the mess that Documentation/ is, and > forces readers to go digging through all kinds of inappropriate material > in the hope of finding something that tells them what they need to know. > > So I would *really* like to split the documentation by audience, as has > been done for a number of other kernel subsystems (and eventually all, I > hope). > > I can go ahead and apply the RST conversion, that seems like a step in > the right direction regardless. But I sure hope we don't really have to > keep it as an unorganized jumble of stuff... I've started on Documentation/core-api/memory.rst which covers just memory allocation. So far it has the Overview and GFP flags sections written and an outline for 'The slab allocator', 'The page allocator', 'The vmalloc allocator' and 'The page_frag allocator'. And typing this up, I realise we need a 'The percpu allocator'. I'm thinking that this is *not* the right document for the DMA memory allocators (although it should link to that documentation). I suspect the existing Documentation/vm/ should probably stay as an unorganised jumble of stuff. Developers mostly talking to other MM developers. Stuff that people outside the MM fraternity should know about needs to be centrally documented. By all means convert it to ReST ... I don't much care, and it may make it easier to steal bits or link to it from the organised documentation. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] COPYING: create a new file with points to the Kernel license files
On Thu, Mar 22, 2018 at 06:54:13AM -0300, Mauro Carvalho Chehab wrote: > +++ b/Documentation/process/license-rules.rst > @@ -4,15 +4,17 @@ Linux kernel licensing rules > > > The Linux Kernel is provided under the terms of the GNU General Public > -License version 2 only (GPL-2.0), as published by the Free Software > -Foundation, and provided in the COPYING file. This documentation file is > -not meant to replace the COPYING file, but provides a description of how > -each source file should be annotated to make the licensing it is governed > -under clear and unambiguous. > - > -The license in the COPYING file applies to the kernel source as a whole, > -though individual source files can have a different license which is > -required to be compatible with the GPL-2.0:: > +version 2 only (GPL-2.0), as written at LICENSES/preferred/GPL-2.0, ^^^ you dropped the word 'License' here Also, I think this should read "as provided in", not "as written at". > +with an explicit syscall exception described at s/at/in/ > +LICENSES/exceptions/Linux-syscall-note, as described in the COPYING file. This phrasing is awkward with "desribed" used twice in the same sentence ... > +This documentation file is not meant to replace the Kernel's license, > +but provides a description of how each source file should be annotated > +to make the licensing it is governed under clear and unambiguous. I'd rather this said: This documentation file provides a description of how each source file should be annotated to make its license clear and unambiguous. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2] docs: add Co-Developed-by docs
On Mon, Mar 05, 2018 at 02:58:21PM +1100, Tobin C. Harding wrote: > -12) When to use Acked-by: and Cc: > -- > +12) When to use Acked-by: and Cc:, and Co-Developed-by: > +--- +12) When to use Acked-by:, Cc:, and Co-Developed-by: -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Documentation/sphinx: Fix Directive import error
On Fri, Mar 02, 2018 at 04:28:31PM +0100, Takashi Iwai wrote: > from docutils.parsers.rst import directives > -from sphinx.util.compat import Directive > +try: > +from docutils.parsers.rst import directives, Directive We also don't need 'directives' on this line as it was imported on the previous line. > +except ImportError: > +from sphinx.util.compat import Directive > from sphinx.ext.autodoc import AutodocReporter > > __version__ = '1.0' Here's what I tested on Debian: +++ b/Documentation/sphinx/kerneldoc.py @@ -37,7 +37,10 @@ import glob from docutils import nodes, statemachine from docutils.statemachine import ViewList from docutils.parsers.rst import directives -from sphinx.util.compat import Directive +try: +from docutils.parsers.rst import Directive +except ImportError: +from sphinx.util.compat import Directive from sphinx.ext.autodoc import AutodocReporter __version__ = '1.0' -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Documentation/sphinx: Fix Directive import error
On Fri, Mar 02, 2018 at 08:42:54AM -0800, Matthew Wilcox wrote: > On Fri, Mar 02, 2018 at 06:01:50PM +0200, Jani Nikula wrote: > > On Fri, 02 Mar 2018, Takashi Iwai <ti...@suse.de> wrote: > > > The sphinx.util.compat for Directive stuff was deprecated in the > > > recent Sphinx version, and now we get a build error. > > > > > > Let's import from the new place, docutils.parsers.rst, while keeping > > > the old sphinx.util.compat as fallback. > > > > > > Bugzilla: https://bugzilla.opensuse.org/show_bug.cgi?id=1083694 > > > Signed-off-by: Takashi Iwai <ti...@suse.de> > > > --- > > > v1->v2: Change the fallback order as Matthew suggested, the new one at > > > first > > > > So this crossed my mind as well... and then I thought it'll probably > > succeed on older Sphinx, and the fallback is not needed. The question > > is, are these equal? Can we just import from docutils.parsers.rst? > > I found a github page which implies that docutils.parsers.rst.Directive > was added 12 years ago (!) so we're probably safe to rely on it: > > https://github.com/docutils-mirror/docutils/commit/9649abee47b4ce4db51be1d90fcb1fb500fa78b3 > > Again, I'm no pythonista, so I may have muddled this. I spent some time delving through the Sphinx codebase. The compat version was added so you could use Directive with docutils 0.4 onwards. docutils 0.5 onwards has Directive. So as long as we're comfortable mandating docutils from 2009, we're fine ;-) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Documentation/sphinx: Fix Directive import error
On Fri, Mar 02, 2018 at 06:01:50PM +0200, Jani Nikula wrote: > On Fri, 02 Mar 2018, Takashi Iwaiwrote: > > The sphinx.util.compat for Directive stuff was deprecated in the > > recent Sphinx version, and now we get a build error. > > > > Let's import from the new place, docutils.parsers.rst, while keeping > > the old sphinx.util.compat as fallback. > > > > Bugzilla: https://bugzilla.opensuse.org/show_bug.cgi?id=1083694 > > Signed-off-by: Takashi Iwai > > --- > > v1->v2: Change the fallback order as Matthew suggested, the new one at first > > So this crossed my mind as well... and then I thought it'll probably > succeed on older Sphinx, and the fallback is not needed. The question > is, are these equal? Can we just import from docutils.parsers.rst? I found a github page which implies that docutils.parsers.rst.Directive was added 12 years ago (!) so we're probably safe to rely on it: https://github.com/docutils-mirror/docutils/commit/9649abee47b4ce4db51be1d90fcb1fb500fa78b3 Again, I'm no pythonista, so I may have muddled this. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3] Documentation/sphinx: Fix Directive import error
From: Matthew Wilcox <mawil...@microsoft.com> Sphinx 1.7 removed sphinx.util.compat.Directive so people who have upgraded cannot build the documentation. Switch to docutils.parsers.rst.Directive which has been available since docutils 0.5 released in 2009. Bugzilla: https://bugzilla.opensuse.org/show_bug.cgi?id=1083694 Co-developed-by: Takashi Iwai <ti...@suse.de> Signed-off-by: Matthew Wilcox <mawil...@microsoft.com> diff --git a/Documentation/sphinx/kerneldoc.py b/Documentation/sphinx/kerneldoc.py index 39aa9e8697cc..fbedcc39460b 100644 --- a/Documentation/sphinx/kerneldoc.py +++ b/Documentation/sphinx/kerneldoc.py @@ -36,8 +36,7 @@ import glob from docutils import nodes, statemachine from docutils.statemachine import ViewList -from docutils.parsers.rst import directives -from sphinx.util.compat import Directive +from docutils.parsers.rst import directives, Directive from sphinx.ext.autodoc import AutodocReporter __version__ = '1.0' -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3] usb: core: Add "quirks" parameter for usbcore
On Sun, Feb 25, 2018 at 08:38:33PM +0800, Kai-Heng Feng wrote: > v2: Use in-kernel tolower() function. ... why are you using tolower at all? You've got 13 quirks already; you may need to use upper case as well before too long. > + quirk = vmalloc(sizeof(struct quirk_entry)); vmalloc? For a struct that's 24 bytes? You know that allocates an entire 4k page, right? -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4] usb: core: Add "quirks" parameter for usbcore
On Mon, Feb 26, 2018 at 11:04:57PM +0800, Kai-Heng Feng wrote: > +static char quirks_param[128]; > +module_param_string(quirks, quirks_param, sizeof(quirks_param), 0644); > +MODULE_PARM_DESC(quirks, "Add/modify USB quirks by specifying > quirks=vendorID:productID:quirks"); > + > +static char quirks_param_orig[128]; > +static u32 usb_detect_dynamic_quirks(struct usb_device *udev) > +{ > + u16 vid = le16_to_cpu(udev->descriptor.idVendor); > + u16 pid = le16_to_cpu(udev->descriptor.idProduct); > + struct quirk_entry *quirk; > + > + mutex_lock(_mutex); > + if (strcmp(quirks_param, quirks_param_orig) != 0) { > + strcpy(quirks_param_orig, quirks_param); What happens if the user is writing to quirks_param at the same time that you memcpy it? I think you're going about this wrong by trying to use the module_param_string machinery. You should be using module_param_cb() to build the quirks list when the user writes it (and then translate back into a string when the user wants to read from it. Also, you won't need to use a linked list for this; you can just allocate an array of quirks. -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4] usb: core: Add "quirks" parameter for usbcore
On Thu, Mar 01, 2018 at 12:01:40AM +0800, Kai Heng Feng wrote: > > Also, you won't need to use a linked list for this; you can just allocate > > an array of quirks. > > I use linked list because the total quirks number is known after the entire > string gets parsed. > Do you suggest that I should just alloc a predefined number (like 16) quirk > entries, instead of doing it dynamically? I'd just do: unsigned int nr = 1; for (i = 0; i < len; i++) if (str[i] == ',') nr++; kcalloc(nr, sizeof(struct), GFP_KERNEL); -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Documentation/sphinx: Fix Directive import error
On Fri, Mar 02, 2018 at 12:49:03PM +0100, Takashi Iwai wrote: > I'm no expert of sphinx nor python, so something might be wrong. > Please check it. I'm also not a pythonista, but ... > --- a/Documentation/sphinx/kerneldoc.py > +++ b/Documentation/sphinx/kerneldoc.py > @@ -37,7 +37,10 @@ import glob > from docutils import nodes, statemachine > from docutils.statemachine import ViewList > from docutils.parsers.rst import directives > -from sphinx.util.compat import Directive > +try: > +from sphinx.util.compat import Directive > +except ImportError: > +from docutils.parsers.rst import directives, Directive It seems to me the previous line already imported docutils.parsers.rst.directives, and we should probably prefer the newer parser even with Sphinx 1.6, so I would think this would work better: -from sphinx.util.compat import Directive +try: +from docutils.parsers.rst import Directive +except ImportError: +from sphinx.util.compat import Directive (it works on Debian with Sphinx 1.6.7) -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: > @@ -21,10 +21,10 @@ Virtual Memory Primer > The physical memory in a computer system is a limited resource and > even for systems that support memory hotplug there is a hard limit on > the amount of memory that can be installed. The physical memory is not > -necessary contiguous, it might be accessible as a set of distinct > +necessary contiguous; it might be accessible as a set of distinct necessarily > address ranges. Besides, different CPU architectures, and even > -different implementations of the same architecture have different view > -how these address ranges defined. > +different implementations of the same architecture have different views > +of how these address ranges defined. "are defined"? > Each physical memory page can be mapped as one or more virtual > pages. These mappings are described by page tables that allow > -translation from virtual address used by programs to real address in > -the physical memory. The page tables organized hierarchically. > +translation from a virtual address used by programs to the real > +address in the physical memory. The page tables are organized > +hierarchically. I don't like the term "real address". Can we say "physical address in memory" here, or "address of physical memory" or something? > -page filled with zeroes. When the program performs a write, regular > +page filled with zeroes. When the program performs a write, a regular > physical page will be allocated to hold the written data. The page > will be marked dirty and if the kernel will decide to repurpose it, > the dirty page will be swapped out. "if the kernel will decide to repurpose it" is awkward. How about if the kernel decides to repurpose it"? > OOM killer > == > > -It may happen, that on a loaded machine memory will be exhausted. When > +It may happen that on a loaded machine memory will be exhausted. When > the kernel detects that the system runs out of memory (OOM) it invokes > `OOM killer`. > Its mission is simple: all it has to do is to select a > task to sacrifice for the sake of the overall system health. The It is possible for the kernel to be unable to reclaim enough memory to continue to operate. In order to save the rest of the system, it invokes the `OOM killer` to sacrifice one task.
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Sat, Oct 06, 2018 at 10:51:54AM +1000, Dave Chinner wrote: > On Wed, Oct 03, 2018 at 09:18:11PM -0700, Darrick J. Wong wrote: > > Hi all, > > > > This series converts the existing in-kernel xfs documentation to rst > > format, links it in with the rest of the kernel's rst documetation, and > > then begins pulling in the contents of the Data Structures & Algorithms > > book from the xfs-documentation git tree. No changes are made to the > > text during the import process except to fix things that the conversion > > process (asciidoctor + pandoc) didn't do correctly. The goal of this > > series is to tie together the XFS code with the on-disk format > > documentation for the features supported by the code. > > > > I've built the docs and put them here, in case you hate reading rst: > > https://djwong.org/docs/kdoc/admin-guide/xfs.html > > https://djwong.org/docs/kdoc/filesystems/xfs-data-structures/index.html > > > > I've posted a branch here because the png import patch is huge: > > https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=docs-4.20-merge > > > > The patchset should apply cleanly against 4.19-rc6. Comments and > > questions are, as always, welcome. > > Jon, > > Can you let us know whether the CC-by-SA 4.0 license is acceptible > or not? That's really the only thing that we need clarified at this > point - if it's OK I'll to pull this into the XFS tree for the 4.20 > merge window. If not, we'll go back to the drawing board This is (obviously) not legal advice. I had an informal chat with Bradley Kuhn at LinuxCon Japan about using CC-BY-SA-4.0 and he indicated that I was probably better off using the GPL-2(+) for documentation. I've changed the XArray documentation from CC-BY-SA-4.0 to GPL-2+. YMMV, defer to the LF lawyer, but I thought it worth providing a data point.
[PATCH] Link the memory allocation guide from the MM docs
I just went looking for the memory allocation guide in the MM docs instead of in the core API. For the benefit of the next person who makes that mistake, link to it from the MM docs. Signed-off-by: Matthew Wilcox diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst index f8bb9aa120c4..8954a88ff5b7 100644 --- a/Documentation/core-api/memory-allocation.rst +++ b/Documentation/core-api/memory-allocation.rst @@ -1,3 +1,5 @@ +.. _memory-allocation: + === Memory Allocation Guide === diff --git a/Documentation/vm/index.rst b/Documentation/vm/index.rst index c4ded22197ca..2b3ab3a1ccf3 100644 --- a/Documentation/vm/index.rst +++ b/Documentation/vm/index.rst @@ -2,7 +2,9 @@ Linux Memory Management Documentation = -This is a collection of documents about Linux memory management (mm) subsystem. +This is a collection of documents about the Linux memory management (mm) +subsystem. If you are looking for advice on simply allocating memory, +see the :ref:`memory-allocation`. User guides for MM features ===