On Wed, 2021-05-19 at 11:22 -0400, Wolfgang E. Sanyer wrote: > On Wed, May 19, 2021 at 11:02 AM Michał Górny <mgo...@gentoo.org> wrote: > > > > On Wed, 2021-05-19 at 10:37 -0400, Wolfgang E. Sanyer wrote: > > > See some comments in-line below. > > > > > > > > > On Wed, May 19, 2021 at 8:33 AM Michał Górny <mgo...@gentoo.org> wrote: > > > > > > > > Hi, > > > > > > > > Please review the pre-GLEP inlined below. Its purpose is to formally > > > > define the format of layout.conf. It's pretty much inevitable these > > > > days, so we should specify it. However, it doesn't really fit into PMS, > > > > and other formats (Manifests, metadata.xml) are already defined > > > > in GLEPs, so this just follows suit. > > > > > > > > Pre-GLEP follows. > > > > > > > > --- > > > > GLEP: 9999 > > > > Title: Repository configuration file (layout.conf) > > > > Author: Michał Górny <mgo...@gentoo.org> > > > > Type: Standards Track > > > > Status: Draft > > > > Version: 1.0 > > > > Created: 2021-05-19 > > > > Last-Modified: 2021-05-19 > > > > Post-History: 2021-05-19 > > > > Content-Type: text/x-rst > > > > --- > > > > > > > > Abstract > > > > ======== > > > > > > > > The ``metadata/layout.conf`` file format is specified as used by Portage > > > > and PkgCore. A standard set of configuration keys is described > > > > including the keys currently used in the Gentoo repository. > > > > > > > > > > > > Motivation > > > > ========== > > > > > > > > The ``metadata/layout.conf`` file was first added to the Gentoo > > > > repository in Oct 2011, to facilitate setting of hashes used > > > > in Manifest2 files. In Mar 2012, it was used to indicate the transition > > > > to the new ``md5-dict`` cache format. In Jul 2013, it started being > > > > used to indicate the repository's masters and effectively became > > > > obligatory for all repositories. > > > > > > > > Today, ``layout.conf`` is used for various repository configuration > > > > knobs that can be expressed as simple values and therefore > > > > do not justify adding new files to the repository. This primarily > > > > involves the configuration of development tools but also includes a few > > > > keys relevant to the behavior of the package manager. > > > > > > > > However, ``layout.conf`` is currently not covered by any formal > > > > specification. The PMS neglects its existence entirely, and the keys > > > > used are roughly defined by their first use of Portage or PkgCore. > > > > This GLEP aims to overcome this by providing a formal specification > > > > for the file, as well as an up-to-date list of permitted configuration > > > > keys. > > > > > > > > > > > > Specification > > > > ============= > > > > > > > > layout.conf file format > > > > ----------------------- > > > > > > > > Every ebuild repository must contain a ``metadata/layout.conf`` file. > > > > The file uses a line-oriented text format. Lines starting with ``#`` > > > > represent comments and are ignored, as are lines consisting entirely > > > > of whitespace. The remaining lines must contain a key followed > > > > by equals sign (``=``), followed by a (possibly empty) value. Each of > > > > these elements can be surrounded by additional whitespace that > > > > is stripped. > > > > > > Is a space around the "=" optional? I see it used below, but here it > > > sounds > > > like there should be no space. > > > > The last sentence says you can add extra space and it will be stripped. > > If you can think of a better way of wording that (I really wanted to > > avoid 'optional whitespace, followed by key, followed by optional > > whitespace... ' ;-). > > 🤦♂️ ofc I would miss that. > > I always thought that python did a pretty good job > [explaining their format mini-language][1]. That's obviously overkill for what > you're describing here, but maybe something like this would be helpful? > > # comments are allowed > <key> = [<value1> [<value2>...]] > > Any lines starting with a ``#`` represents a comment and are ignored, as are > lines consisting entirely of whitespace. > > A valid line consists of key and an equal sign. (A valid key is a > string containing > any character except for a space)[^1]. The list of space-separated > values is optional. > > Note: any additional spaces are ultimately stripped > > [1]: > https://docs.python.org/3/library/string.html#format-specification-mini-language > [^1]: Should we include this or nah?
Thanks, this looks like a nice idea. I'm going to send a v2 shortly and we'll see how it worked out. > > > > > > > > Configuration keys > > > > ------------------ > > > > > > > > The ``layout.conf`` file must contain the ``masters`` key. Other keys > > > > listed in this specification are entirely optional. The package > > > > managers may choose to implement a subset of listed keys. Unknown keys > > > > must be ignored. > > > > > > > > The following keys are currently defined: > > > > > > Is the intent for the GLEP to define and specify every possible key-value > > > combination? If not, then perhaps this section should be truncated a bit? > > > i.e. > > > "here are some examples of currently defined keys" and then list a > > > handful. > > > > > > Additionally, maybe the spec should include some sort of requirement for > > > documentation of variables, similar to what you've done below. > > > > > > Edit: ok, nevermind I see your recommendation at the bottom. But still, > > > I would probably either (1) insist that new keys be added to this GLEP, or > > > (2) insist that new keys are documented properly. > > > > Let's put it like this: the goal is to document all the useful keys. > > I can live with some ancient-not-really-useful keys not being documented > > (e.g. Portage has some stale/redundant Manifest-related keys, I think). > > Perhaps then some sort of use of the word "reserved" should be in here? > i.e. "The following keys are reserved for use by portage and pkgcheck. > any other keys may be defined and used at will" > > > > > > > > masters = <space-separated values> > > > > Specifies the master repositories of this repository. For stand-alone > > > > repositories, this must be set to an empty value. Otherwise, it can > > > > list one or more repositories, separated by spaces. This key must > > > > be specified. > > > > > > I realize this is the only compulsory key, but perhaps it should still go > > > in > > > a separate section in order to ensure that it stands out? i.e. "The > > > following > > > keys are mandory..." and then later "the following optional keys are > > > currently defined" > > > > I'm not sure, to be honest. I didn't really want to add a section for > > one key ;-). > > np, just a thought. > > > > > > > > manifest-hashes = <space-separated values> > > > > Specifies the list of hashes that should be used for new distfiles > > > > in the Manifest files. The development tools may create a subset > > > > of the specified hashes if it is not updating the checksums for > > > > the specified distfile, or does not support the hash in question. > > > > The hash names are specified in GLEP 74. [#GLEP74]_ The default > > > > set of hashes is implementation-defined. > > > > > > > > manifest-required-hashes = <space-separated values> > > > > Specifies the list of hashes that must be used in Manifest files. > > > > The development tools must support all the hashes listed there, > > > > and update distfile checksums to use these hashes (refetching > > > > if necessary). This must be a subset of manifest-hashes. If not > > > > specified, all hashes from manifest-hashes (or the default set) > > > > are considered required. > > > > > > > > use-manifests = ``strict``, ``true`` or ``false`` > > > > Indicates the policy for creating and using Manifest files. If set > > > > to ``strict``, Manifest files are created and files are required to > > > > match digests found in Manifests. If set to ``true``, Manifests > > > > are created but digest mismatches are ignored. If set to ``false``, > > > > Manifests are not used at all. The default is ``strict``. > > > > > > > > update-changelog = ``true`` or ``false`` > > > > Indicates whether the development tools should write ChangeLog files. > > > > The default is ``false``. > > > > > > > > cache-formats = <space-separated values> > > > > Specifies one or more cache formats used by the repository. > > > > The currently defined values are ``pms`` for the original format > > > > specified in the PMS and ``md5-dict`` for the md5-dict format > > > > introduced in Portage 2.2.0_alpha68. The default is > > > > implementation-defined. > > > > > > > > eapis-deprecated = <space-separated values> > > > > Specifies one or more EAPIs that are to be considered deprecated > > > > by the development tools for use in ebuilds, i.e. their use should > > > > trigger a warning. If not specified, no EAPIs are deprecated. > > > > > > > > eapis-banned = <space-separated values> > > > > Specifies one or more EAPIs that are to be considered banned > > > > by the development tools for use in ebuilds, i.e. their use should > > > > be blocked. If not specified, no EAPIs are banned. > > > > > > > > repo-name = <string> > > > > Specifies the repository name. If specified, it must be equal > > > > to the contents of ``profiles/repo_name``. If not specified, > > > > it defaults to the same value. Discouraged. > > > > > > > > aliases = <space-separated values> > > > > Specified one or more additional names that can be used to reference > > > > the repository (e.g. in repository dependencies). If not specified, > > > > no aliases are defined. > > > > > > > > thin-manifests = ``true`` or ``false`` > > > > If enabled, Manifest files in the package directory must contain only > > > > ``DIST`` entries. If disabled, Manifest files in the package > > > > directory must list digests for all files found in the package > > > > directory and the files directory. The default is ``false``. > > > > > > > > sign-commits = ``true`` or ``false`` > > > > Indicates whether git commits are to be signed (using ``git commit > > > > --gpg-sign``. The default is ``false``. > > > > > > > > sign-manifests = ``true`` or ``false`` > > > > Indicates whether individual package Manifests should be PGP-signed. > > > > Note that this refers to the historical behavior of signing individual > > > > Manifests, not the GLEP 74 behavior of signing the top-level Manifest. > > > > [#GLEP74]_ The default is ``true`` if PGP signing is configured. > > > > > > > > properties-allowed = <space-separated values> > > > > Specifies the list of ``PROPERTIES`` tokens that are permitted > > > > to be used in ebuilds. If present, the development tools should issue > > > > a warning if ``PROPERTIES`` contains any tokens that are not listed > > > > here. If not specified, all tokens are permitted. > > > > > > > > restrict-allowed = <space-separated values> > > > > Same as properties-allowed, except for ``RESTRICT``. > > > > > > > > profile-formats = <space-separated values> > > > > Specifies the format used by profiles and/or extensions to it. > > > > The default is ``pms`` indicating the format specified in the PMS. > > > > Other values are implementation-defined. > > > > > > > > > > > > It is recommended that any future keys are added to this GLEP before > > > > being implemented. > > > > > > > > > > > > Example > > > > ------- > > > > > > > > The following is an example configuration for a git repository with > > > > Gentoo set as a master:: > > > > > > > > masters = gentoo > > > > > > > > # git: do not use ChangeLog, use thin, unsigned Manifests > > > > update-changelog = false > > > > thin-manifests = true > > > > sign-manifests = false > > > > > > > > # force the new md5-dict cache format > > > > cache-formats = md5-dict > > > > > > > > > > > > Rationale > > > > ========= > > > > > > > > This GLEP is written almost 10 years after ``layout.conf`` was > > > > originally introduced. This made it necessary to write it in such a way > > > > that both the modern and historical implementations in Portage > > > > and PkgCore, as well as the use in the Gentoo repository > > > > and a reasonably large subset of the other repositories would remain > > > > compliant. > > > > > > > > The historical default of assuming ``masters = gentoo`` when unspecified > > > > is omitted as it is not portable and verbosely deprecated for many > > > > years in Portage. All repositories are required to explicitly specify > > > > their masters, or an empty value if they are stand-alone. > > > > > > > > The default for Manifest hashes and cache formats are left to be > > > > implementation-defined, as the defaults changed over time and do not > > > > match between package managers. In particular, Portage attempts to > > > > autodetect the cache format currently used in a given repository. > > > > > > > > The repo-name key has been originally added as an alternative to > > > > ``profiles/repo_name``. However, the latter file is still required > > > > for PMS compliance. Furthermore, given that it is much easier to parse, > > > > there seems to be no appealing reason to work towards replacing that > > > > file. This means that for all practical reasons, the repo-name key > > > > is redundant and is listed here for completeness only. > > > > > > > > The profile-formats key has been introduced to permit Portage-specific > > > > extensions to the profile directory without having to introduce custom > > > > EAPIs. The exact extensions are considered outside the scope of this > > > > specification. > > > > > > > > > > > > Backwards Compatibility > > > > ======================= > > > > > > > > The existing implementations found in Portage and PkgCore conform > > > > to this specification, so does the ``metadata/layout.conf`` file > > > > found in the Gentoo repository. > > > > > > > > > > > > Reference Implementation > > > > ======================== > > > > > > > > The support for ``metadata/layout.conf`` is already a part of Portage > > > > and PkgCore. > > > > > > > > > > > > References > > > > ========== > > > > > > > > .. [#GLEP74] GLEP 74: Full-tree verification using Manifest files > > > > (https://www.gentoo.org/glep/glep-0074.html) > > > > > > > > > > > > Copyright > > > > ========= > > > > > > > > This work is licensed under the Creative Commons Attribution-ShareAlike > > > > 4.0 > > > > International License. To view a copy of this license, visit > > > > https://creativecommons.org/licenses/by-sa/4.0/. > > > > > > > > > > > > -- > > > > Best regards, > > > > Michał Górny > > > > > > > > > > > > > > > > > > > -- > > Best regards, > > Michał Górny > > > > > > > -- Best regards, Michał Górny