[gentoo-dev] Last-rites: app-dicts/qvortaro

2018-01-30 Thread Andreas Sturmlechner 
# Andreas Sturmlechner  (31 Jan 2018)
# No homepage, depends on deprecated Qt4.
# Masked for removal in 30 days.
app-dicts/qvortaro

[gentoo-dev] Last-rites: media-sound/qpitch

2018-01-30 Thread Andreas Sturmlechner 
# Andreas Sturmlechner  (31 Jan 2018)
# Dead upstream, depends on deprecated Qt4.
# Masked for removal in 30 days. Bug #644516
media-sound/qpitch

[gentoo-dev] Last-rites: media-sound/qtagconvert

2018-01-30 Thread Andreas Sturmlechner 
# Andreas Sturmlechner  (31 Jan 2018)
# Dead upstream, depends on deprecated Qt4.
# Masked for removal in 30 days. Bug #646182
media-sound/qtagconvert

Re: [gentoo-dev] [PATCH] use.desc: Correct/clarify SSL/TLS-related flags

2018-01-30 Thread Gordon Pettey 
On Tue, Jan 30, 2018 at 5:22 PM, Ulrich Mueller  wrote:
>> On Tue, 30 Jan 2018, Michał Górny wrote:
> NACK. This seems to imply that USE="-ssl gnutls" is not a valid
> configuration? What if the user prefers gnutls and therefore has
> globally enabled the gnutls flag, but -ssl for a single package?

Because having gnutls enabled and ssl disabled, if a package has both
flags, is nonsense? What is "I want gnutls but I don't want support
for SSL/TLS" supposed to do?

Re: [gentoo-dev] [PATCH] use.desc: Correct/clarify SSL/TLS-related flags

2018-01-30 Thread Kristian Fiskerstrand 
On 01/31/2018 12:22 AM, Ulrich Mueller wrote:
>>  gnome-keyring - Enable support for storing passwords via gnome-keyring
>>  gnuplot - Enable support for gnuplot (data and function plotting)
>> -gnutls - Add support for net-libs/gnutls (TLS 1.0 and SSL 3.0 support)
>> +gnutls - Prefer net-libs/gnutls as SSL/TLS provider (requires USE=ssl if 
>> present)
> NACK. This seems to imply that USE="-ssl gnutls" is not a valid
> configuration? What if the user prefers gnutls and therefore has
> globally enabled the gnutls flag, but -ssl for a single package?
> 
> How about "(needs USE=ssl to take effect)" instead?
> 

as I understand it ssl is intended as a generic use flag, of which
gnutls can be one of the providers. In the case of of app-crypt/gnupg
there are only two possible providers, gnutls, and ntbtls, of which only
one is available in tree, so gnutls is the only one, so the only one
relevant for Gentoo is gnutls, hence no use flag for it, either TLS is
enabled, or it is not.

in this scenario I don't see why "ssl -gnutls" would not be a valid
configuration as long as ssl is a generic use flag as it is presented to
be. It doesn't mean never install gnutls, but just not preferring it in
cases where there are other providers of ssl/tls, that the global
description already indicate.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [PATCH] use.desc: Correct/clarify SSL/TLS-related flags

2018-01-30 Thread Ulrich Mueller 
> On Tue, 30 Jan 2018, Michał Górny wrote:

> Correct the description of SSL/TLS-related flags to match their modern
> use. USE=ssl is a feature flag that enables support for SSL/TLS,
> while USE=gnutls and USE=libressl are implementation toggling flags.

> Unify the descriptions a bit. Make sure to mention both SSL and TLS
> to avoid confusion. Inform about the necessity of enabling USE=ssl
> in both implementation flags, and replace 'might' with 'if present'.
> ---
>  profiles/use.desc | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)

> diff --git a/profiles/use.desc b/profiles/use.desc
> index 43423a017a5f..4ac5d5ad6136 100644
> --- a/profiles/use.desc
> +++ b/profiles/use.desc
> @@ -119,7 +119,7 @@ gmp - Add support for dev-libs/gmp (GNU MP library)
>  gnome - Add GNOME support
>  gnome-keyring - Enable support for storing passwords via gnome-keyring
>  gnuplot - Enable support for gnuplot (data and function plotting)
> -gnutls - Add support for net-libs/gnutls (TLS 1.0 and SSL 3.0 support)
> +gnutls - Prefer net-libs/gnutls as SSL/TLS provider (requires USE=ssl if 
> present)

NACK. This seems to imply that USE="-ssl gnutls" is not a valid
configuration? What if the user prefers gnutls and therefore has
globally enabled the gnutls flag, but -ssl for a single package?

How about "(needs USE=ssl to take effect)" instead?

>  gphoto2 - Add digital camera support
>  gpm - Add support for sys-libs/gpm (Console-based mouse driver)
>  gps - Add support for Global Positioning System
> @@ -179,7 +179,7 @@ libcaca - Add support for colored ASCII-art graphics
>  libedit - Use the libedit library (replacement for readline)
>  libffi - Enable support for Foreign Function Interface library
>  libnotify - Enable desktop notification support
> -libressl - Use dev-libs/libressl as SSL provider (might need ssl USE flag), 
> packages should not depend on this USE flag
> +libressl - Use dev-libs/libressl instead of dev-libs/openssl as SSL/TLS 
> provider (requires USE=ssl if present), packages should not depend on this 
> USE flag

Same here.

>  libsamplerate - Build with support for converting sample rates using 
> libsamplerate
>  libwww - Add libwww support (General purpose WEB API)
>  lirc - Add support for lirc (Linux's Infra-Red Remote Control)
> @@ -319,7 +319,7 @@ sox - Add support for Sound eXchange (SoX)
>  speex - Add support for the speex audio codec (used for speech)
>  spell - Add dictionary support
>  sqlite - Add support for sqlite - embedded sql database
> -ssl - Add support for Secure Socket Layer connections
> +ssl - Add support for SSL/TLS connections (Secure Socket Layer / Transport 
> Layer Security)
>  startup-notification - Enable application startup event feedback mechanism
>  static - !!do not set this during bootstrap!! Causes binaries to be 
> statically linked instead of dynamically
>  static-libs - Build static versions of dynamic libraries as well
> -- 
> 2.16.1


pgpnVRtA9g04Q.pgp
Description: PGP signature

Re: [gentoo-dev] [PATCH] use.desc: Correct/clarify SSL/TLS-related flags

2018-01-30 Thread Kristian Fiskerstrand 
On 01/30/2018 11:11 PM, Michał Górny wrote:
> Correct the description of SSL/TLS-related flags to match their modern
> use. USE=ssl is a feature flag that enables support for SSL/TLS,
> while USE=gnutls and USE=libressl are implementation toggling flags.
> 
> Unify the descriptions a bit. Make sure to mention both SSL and TLS
> to avoid confusion. Inform about the necessity of enabling USE=ssl
> in both implementation flags, and replace 'might' with 'if present'.
> 
+1 / Reviewed-By

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v4] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Michał Górny 
W dniu wto, 30.01.2018 o godzinie 14∶10 -0800, użytkownik Zac Medico
napisał:
> On 01/30/2018 11:02 AM, Michał Górny wrote:
> > Warn about empty directories installed to /var in install-qa-check phase
> > (that were not "filled" using keepdir), to help developers stop relying
> > upon Portage preserving them. Those directories are rather unlikely to
> > be false positives.
> > 
> > Furthermore, remove all the empty directories if FEATURES=strict-keepdir
> > is used to catch even more problems (intended for developers). Here
> > warnings are not really suitable since there will be a high number
> > of false positives.
> > 
> > The PMS specifies the behavior upon merging empty directories
> > as undefined, and specifically prohibits ebuilds from attempting
> > to install empty directories. However, ebuilds occasionally still fall
> > into the trap of relying on 'dodir' preserving the directory. Make
> > the Portage behavior more strict in order to prevent that.
> > 
> > [v4: now with LC_COLLATE=C for sorting]
> > ---
> >  bin/install-qa-check.d/95empty-dirs | 42 
> > +
> >  man/make.conf.5 |  4 
> >  pym/portage/const.py|  1 +
> >  3 files changed, 47 insertions(+)
> >  create mode 100644 bin/install-qa-check.d/95empty-dirs
> > 
> > diff --git a/bin/install-qa-check.d/95empty-dirs 
> > b/bin/install-qa-check.d/95empty-dirs
> > new file mode 100644
> > index 0..8599db395
> > --- /dev/null
> > +++ b/bin/install-qa-check.d/95empty-dirs
> > @@ -0,0 +1,42 @@
> > +# Warn about and/or remove empty directories installed by ebuild.
> > +
> > +# Rationale: PMS prohibits ebuilds from installing empty directories.
> > +# Cleaning them up from the installation image provides an easy way
> > +# to make sure that ebuilds are not relying on it while making it easy
> > +# for users to override this if they need to.
> > +#
> > +# The ebuilds that need to preserve empty directories should use keepdir
> > +# as documented e.g.:
> > +# 
> > https://devmanual.gentoo.org/function-reference/install-functions/index.html
> > +#
> > +# For now, we emit QA warnings for empty directories in /var.
> > +# Additionally, if FEATURES=strict-keepdir is enabled we explicitly
> > +# remove *all* empty directories to trigger breakage.
> > +
> > +find_empty_dirs() {
> > +   local warn_dirs=()
> > +   local d striparg=
> > +
> > +   [[ ${FEATURES} == *strict-keepdir* ]] && striparg=-delete
> > +
> > +   while IFS= read -r -d $'\0' d; do
> > +   [[ ${d} == ${ED%/}/var/* ]] && warn_dirs+=( "${d}" )
> > +   done < <(find "${ED}" -depth -mindepth 1 -type d -empty -print0 
> > ${striparg} | LC_COLLATE=C sort -z)
> > +
> > +   if [[ ${warn_dirs[@]} ]]; then
> > +   eqawarn "One or more empty directories installed to /var:"
> > +   eqawarn
> > +   for d in "${warn_dirs[@]}"; do
> > +   eqawarn "  ${d#${ED%/}}"
> > +   done
> > +   eqawarn
> > +   eqawarn "If those directories need to be preserved, please make 
> > sure to create"
> > +   eqawarn "or mark them for keeping using 'keepdir'. Future 
> > versions of Portage"
> > +   eqawarn "will strip empty directories from installation image."
> > +   fi
> > +}
> > +
> > +find_empty_dirs
> > +: # guarantee successful exit
> > +
> > +# vim:ft=sh
> > diff --git a/man/make.conf.5 b/man/make.conf.5
> > index a81b497bd..cb0f00237 100644
> > --- a/man/make.conf.5
> > +++ b/man/make.conf.5
> > @@ -623,6 +623,10 @@ see \fBinstallsources\fR.
> >  Have portage react strongly to conditions that have the potential to be
> >  dangerous (like missing or incorrect digests for ebuilds).
> >  .TP
> > +.B strict-keepdir
> > +Have portage strictly require keepdir calls in ebuilds.  Empty
> > +directories installed without explicit keepdir will be removed.
> > +.TP
> >  .B stricter
> >  Have portage react strongly to conditions that may conflict with system
> >  security provisions (for example textrels, executable stack).  Read about
> > diff --git a/pym/portage/const.py b/pym/portage/const.py
> > index e5fa4b67c..655be82b1 100644
> > --- a/pym/portage/const.py
> > +++ b/pym/portage/const.py
> > @@ -184,6 +184,7 @@ SUPPORTED_FEATURES   = frozenset([
> > "split-elog",
> > "split-log",
> > "strict",
> > +   "strict-keepdir",
> > "stricter",
> > "suidctl",
> > "test",
> > 
> 
> Looks good, please merge.

Merged, thanks.
-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] [PATCH] sync-rsync-openpgp-key-path -> generic sync-openpgp-key-path

2018-01-30 Thread Zac Medico 
On 01/30/2018 11:40 AM, Michał Górny wrote:
> All three merged, thanks! Please let me know if directory check also
> looks good, or if you prefer to do it post-release.

It looks good, you can merge it before release.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] [PATCH] use.desc: Correct/clarify SSL/TLS-related flags

2018-01-30 Thread Michał Górny 
Correct the description of SSL/TLS-related flags to match their modern
use. USE=ssl is a feature flag that enables support for SSL/TLS,
while USE=gnutls and USE=libressl are implementation toggling flags.

Unify the descriptions a bit. Make sure to mention both SSL and TLS
to avoid confusion. Inform about the necessity of enabling USE=ssl
in both implementation flags, and replace 'might' with 'if present'.
---
 profiles/use.desc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/profiles/use.desc b/profiles/use.desc
index 43423a017a5f..4ac5d5ad6136 100644
--- a/profiles/use.desc
+++ b/profiles/use.desc
@@ -119,7 +119,7 @@ gmp - Add support for dev-libs/gmp (GNU MP library)
 gnome - Add GNOME support
 gnome-keyring - Enable support for storing passwords via gnome-keyring
 gnuplot - Enable support for gnuplot (data and function plotting)
-gnutls - Add support for net-libs/gnutls (TLS 1.0 and SSL 3.0 support)
+gnutls - Prefer net-libs/gnutls as SSL/TLS provider (requires USE=ssl if 
present)
 gphoto2 - Add digital camera support
 gpm - Add support for sys-libs/gpm (Console-based mouse driver)
 gps - Add support for Global Positioning System
@@ -179,7 +179,7 @@ libcaca - Add support for colored ASCII-art graphics
 libedit - Use the libedit library (replacement for readline)
 libffi - Enable support for Foreign Function Interface library
 libnotify - Enable desktop notification support
-libressl - Use dev-libs/libressl as SSL provider (might need ssl USE flag), 
packages should not depend on this USE flag
+libressl - Use dev-libs/libressl instead of dev-libs/openssl as SSL/TLS 
provider (requires USE=ssl if present), packages should not depend on this USE 
flag
 libsamplerate - Build with support for converting sample rates using 
libsamplerate
 libwww - Add libwww support (General purpose WEB API)
 lirc - Add support for lirc (Linux's Infra-Red Remote Control)
@@ -319,7 +319,7 @@ sox - Add support for Sound eXchange (SoX)
 speex - Add support for the speex audio codec (used for speech)
 spell - Add dictionary support
 sqlite - Add support for sqlite - embedded sql database
-ssl - Add support for Secure Socket Layer connections
+ssl - Add support for SSL/TLS connections (Secure Socket Layer / Transport 
Layer Security)
 startup-notification - Enable application startup event feedback mechanism
 static - !!do not set this during bootstrap!! Causes binaries to be statically 
linked instead of dynamically
 static-libs - Build static versions of dynamic libraries as well
-- 
2.16.1

Re: [gentoo-portage-dev] [PATCH v4] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Zac Medico 
On 01/30/2018 11:02 AM, Michał Górny wrote:
> Warn about empty directories installed to /var in install-qa-check phase
> (that were not "filled" using keepdir), to help developers stop relying
> upon Portage preserving them. Those directories are rather unlikely to
> be false positives.
> 
> Furthermore, remove all the empty directories if FEATURES=strict-keepdir
> is used to catch even more problems (intended for developers). Here
> warnings are not really suitable since there will be a high number
> of false positives.
> 
> The PMS specifies the behavior upon merging empty directories
> as undefined, and specifically prohibits ebuilds from attempting
> to install empty directories. However, ebuilds occasionally still fall
> into the trap of relying on 'dodir' preserving the directory. Make
> the Portage behavior more strict in order to prevent that.
> 
> [v4: now with LC_COLLATE=C for sorting]
> ---
>  bin/install-qa-check.d/95empty-dirs | 42 
> +
>  man/make.conf.5 |  4 
>  pym/portage/const.py|  1 +
>  3 files changed, 47 insertions(+)
>  create mode 100644 bin/install-qa-check.d/95empty-dirs
> 
> diff --git a/bin/install-qa-check.d/95empty-dirs 
> b/bin/install-qa-check.d/95empty-dirs
> new file mode 100644
> index 0..8599db395
> --- /dev/null
> +++ b/bin/install-qa-check.d/95empty-dirs
> @@ -0,0 +1,42 @@
> +# Warn about and/or remove empty directories installed by ebuild.
> +
> +# Rationale: PMS prohibits ebuilds from installing empty directories.
> +# Cleaning them up from the installation image provides an easy way
> +# to make sure that ebuilds are not relying on it while making it easy
> +# for users to override this if they need to.
> +#
> +# The ebuilds that need to preserve empty directories should use keepdir
> +# as documented e.g.:
> +# 
> https://devmanual.gentoo.org/function-reference/install-functions/index.html
> +#
> +# For now, we emit QA warnings for empty directories in /var.
> +# Additionally, if FEATURES=strict-keepdir is enabled we explicitly
> +# remove *all* empty directories to trigger breakage.
> +
> +find_empty_dirs() {
> + local warn_dirs=()
> + local d striparg=
> +
> + [[ ${FEATURES} == *strict-keepdir* ]] && striparg=-delete
> +
> + while IFS= read -r -d $'\0' d; do
> + [[ ${d} == ${ED%/}/var/* ]] && warn_dirs+=( "${d}" )
> + done < <(find "${ED}" -depth -mindepth 1 -type d -empty -print0 
> ${striparg} | LC_COLLATE=C sort -z)
> +
> + if [[ ${warn_dirs[@]} ]]; then
> + eqawarn "One or more empty directories installed to /var:"
> + eqawarn
> + for d in "${warn_dirs[@]}"; do
> + eqawarn "  ${d#${ED%/}}"
> + done
> + eqawarn
> + eqawarn "If those directories need to be preserved, please make 
> sure to create"
> + eqawarn "or mark them for keeping using 'keepdir'. Future 
> versions of Portage"
> + eqawarn "will strip empty directories from installation image."
> + fi
> +}
> +
> +find_empty_dirs
> +: # guarantee successful exit
> +
> +# vim:ft=sh
> diff --git a/man/make.conf.5 b/man/make.conf.5
> index a81b497bd..cb0f00237 100644
> --- a/man/make.conf.5
> +++ b/man/make.conf.5
> @@ -623,6 +623,10 @@ see \fBinstallsources\fR.
>  Have portage react strongly to conditions that have the potential to be
>  dangerous (like missing or incorrect digests for ebuilds).
>  .TP
> +.B strict-keepdir
> +Have portage strictly require keepdir calls in ebuilds.  Empty
> +directories installed without explicit keepdir will be removed.
> +.TP
>  .B stricter
>  Have portage react strongly to conditions that may conflict with system
>  security provisions (for example textrels, executable stack).  Read about
> diff --git a/pym/portage/const.py b/pym/portage/const.py
> index e5fa4b67c..655be82b1 100644
> --- a/pym/portage/const.py
> +++ b/pym/portage/const.py
> @@ -184,6 +184,7 @@ SUPPORTED_FEATURES   = frozenset([
>   "split-elog",
>   "split-log",
>   "strict",
> + "strict-keepdir",
>   "stricter",
>   "suidctl",
>   "test",
> 

Looks good, please merge.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] [pre-GLEP] Split distfile mirror directory structure

2018-01-30 Thread Kent Fredric 
On Tue, 30 Jan 2018 08:25:28 +0100
Michał Górny  wrote:

> W dniu wto, 30.01.2018 o godzinie 14∶21 +1300, użytkownik Kent Fredric
> napisał:
> > On Sat, 27 Jan 2018 12:41:58 +0100
> > Michał Górny  wrote:
> >   
> > >   find -name 'foo.tar.gz'  
> > 
> > Other than being *worse* than the current "ls" situation due to the
> > existence of distfiles/git3-src/ and distfiles/git-src/ 
> >   
> 
> Wait... so people actually don't override those locations?
> 

I'm installing "foo-.ebuild". Why would I read its ebuild, and then
read its eclass, and then, read the documentation on that eclass, and
then, override its defaults?


pgpH__vKJuWpb.pgp
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] sync-rsync-openpgp-key-path -> generic sync-openpgp-key-path

2018-01-30 Thread Michał Górny 
W dniu wto, 30.01.2018 o godzinie 11∶37 -0800, użytkownik Zac Medico
napisał:
> On 01/30/2018 10:56 AM, Michał Górny wrote:
> > W dniu wto, 30.01.2018 o godzinie 09∶32 -0800, użytkownik Zac Medico
> > napisał:
> > > On 01/30/2018 08:59 AM, Michał Górny wrote:
> > > > Rename the 'sync-rsync-openpgp-key-path' to a more generic
> > > > 'sync-openpgp-key-path'. OpenPGP is the basis of at least three
> > > > different verification schemes (git, rsync, snapshots) and at least
> > > > two of them use the same keys.
> > > > ---
> > > >  cnf/repos.conf | 2 +-
> > > >  man/portage.5  | 9 +
> > > >  pym/portage/repository/config.py   | 4 
> > > >  pym/portage/sync/modules/rsync/__init__.py | 1 -
> > > >  pym/portage/sync/modules/rsync/rsync.py| 8 ++--
> > > >  5 files changed, 12 insertions(+), 12 deletions(-)
> > > > 
> > > > diff --git a/cnf/repos.conf b/cnf/repos.conf
> > > > index 0d2b1f4be..4a40ff4fc 100644
> > > > --- a/cnf/repos.conf
> > > > +++ b/cnf/repos.conf
> > > > @@ -7,7 +7,7 @@ sync-type = rsync
> > > >  sync-uri = rsync://rsync.gentoo.org/gentoo-portage
> > > >  auto-sync = yes
> > > >  sync-rsync-verify-metamanifest = yes
> > > > -sync-rsync-openpgp-key-path = 
> > > > /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
> > > > +sync-openpgp-key-path = 
> > > > /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
> > > >  
> > > >  # for daily squashfs snapshots
> > > >  #sync-type = squashdelta
> > > > diff --git a/man/portage.5 b/man/portage.5
> > > > index 84999bd2f..1f6259715 100644
> > > > --- a/man/portage.5
> > > > +++ b/man/portage.5
> > > > @@ -1071,10 +1071,11 @@ Extra options to give to rsync on repository 
> > > > synchronization. It takes
> > > >  precedence over a declaration in [DEFAULT] section, that takes
> > > >  precedence over PORTAGE_RSYNC_EXTRA_OPTS.
> > > >  .TP
> > > > -.B sync\-rsync\-openpgp\-key\-path
> > > > -Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
> > > > -if \fBsync\-rsync\-verify\-metamanifest\fR is enabled. If unset,
> > > > -the user's keyring is used.
> > > > +.B sync\-openpgp\-key\-path
> > > > +Path to the OpenPGP key(ring) used to verify received repository. Used
> > > > +only for protocols supporting cryptographic verification, provided
> > > > +that the respective verification option is enabled. If unset, the 
> > > > user's
> > > > +keyring is used.
> > > >  .TP
> > > >  .B sync-rsync-vcs-ignore = true|false
> > > >  Ignore vcs directories that may be present in the repository. It is the
> > > > diff --git a/pym/portage/repository/config.py 
> > > > b/pym/portage/repository/config.py
> > > > index be31ed3b1..d3a622f7c 100644
> > > > --- a/pym/portage/repository/config.py
> > > > +++ b/pym/portage/repository/config.py
> > > > @@ -86,6 +86,7 @@ class RepoConfig(object):
> > > > 'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 
> > > > 'thin_manifest',
> > > > 'update_changelog', '_eapis_banned', 
> > > > '_eapis_deprecated',
> > > > '_masters_orig', 'module_specific_options', 
> > > > 'manifest_required_hashes',
> > > > +   'openpgp_key_path',
> > > > )
> > > >  
> > > > def __init__(self, name, repo_opts, local_config=True):
> > > > @@ -182,6 +183,9 @@ class RepoConfig(object):
> > > > self.strict_misc_digests = repo_opts.get(
> > > > 'strict-misc-digests', 'true').lower() == 'true'
> > > >  
> > > > +   self.openpgp_key_path = repo_opts.get(
> > > > +   'sync-openpgp-key-path', None)
> > > > +
> > > > self.module_specific_options = {}
> > > >  
> > > > # Not implemented.
> > > > diff --git a/pym/portage/sync/modules/rsync/__init__.py 
> > > > b/pym/portage/sync/modules/rsync/__init__.py
> > > > index 14af2120c..27a2548c0 100644
> > > > --- a/pym/portage/sync/modules/rsync/__init__.py
> > > > +++ b/pym/portage/sync/modules/rsync/__init__.py
> > > > @@ -27,7 +27,6 @@ module_spec = {
> > > > 'validate_config': CheckSyncConfig,
> > > > 'module_specific_options': (
> > > > 'sync-rsync-extra-opts',
> > > > -   'sync-rsync-openpgp-key-path',
> > > > 'sync-rsync-vcs-ignore',
> > > > 'sync-rsync-verify-jobs',
> > > > 'sync-rsync-verify-metamanifest',
> > > > diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> > > > b/pym/portage/sync/modules/rsync/rsync.py
> > > > index 552ac6f6b..d9d7d56f2 100644
> > > > --- a/pym/portage/sync/modules/rsync/rsync.py
> > > > +++ b/pym/portage/sync/modules/rsync/rsync.py
> > > > @@ -87,10 +87,6 @@ class RsyncSync(NewBase):
> > > > self.verify_metamanifest = (
> > > >

Re: [gentoo-portage-dev] [PATCH] sync-rsync-openpgp-key-path -> generic sync-openpgp-key-path

2018-01-30 Thread Zac Medico 
On 01/30/2018 10:56 AM, Michał Górny wrote:
> W dniu wto, 30.01.2018 o godzinie 09∶32 -0800, użytkownik Zac Medico
> napisał:
>> On 01/30/2018 08:59 AM, Michał Górny wrote:
>>> Rename the 'sync-rsync-openpgp-key-path' to a more generic
>>> 'sync-openpgp-key-path'. OpenPGP is the basis of at least three
>>> different verification schemes (git, rsync, snapshots) and at least
>>> two of them use the same keys.
>>> ---
>>>  cnf/repos.conf | 2 +-
>>>  man/portage.5  | 9 +
>>>  pym/portage/repository/config.py   | 4 
>>>  pym/portage/sync/modules/rsync/__init__.py | 1 -
>>>  pym/portage/sync/modules/rsync/rsync.py| 8 ++--
>>>  5 files changed, 12 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/cnf/repos.conf b/cnf/repos.conf
>>> index 0d2b1f4be..4a40ff4fc 100644
>>> --- a/cnf/repos.conf
>>> +++ b/cnf/repos.conf
>>> @@ -7,7 +7,7 @@ sync-type = rsync
>>>  sync-uri = rsync://rsync.gentoo.org/gentoo-portage
>>>  auto-sync = yes
>>>  sync-rsync-verify-metamanifest = yes
>>> -sync-rsync-openpgp-key-path = 
>>> /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
>>> +sync-openpgp-key-path = 
>>> /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
>>>  
>>>  # for daily squashfs snapshots
>>>  #sync-type = squashdelta
>>> diff --git a/man/portage.5 b/man/portage.5
>>> index 84999bd2f..1f6259715 100644
>>> --- a/man/portage.5
>>> +++ b/man/portage.5
>>> @@ -1071,10 +1071,11 @@ Extra options to give to rsync on repository 
>>> synchronization. It takes
>>>  precedence over a declaration in [DEFAULT] section, that takes
>>>  precedence over PORTAGE_RSYNC_EXTRA_OPTS.
>>>  .TP
>>> -.B sync\-rsync\-openpgp\-key\-path
>>> -Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
>>> -if \fBsync\-rsync\-verify\-metamanifest\fR is enabled. If unset,
>>> -the user's keyring is used.
>>> +.B sync\-openpgp\-key\-path
>>> +Path to the OpenPGP key(ring) used to verify received repository. Used
>>> +only for protocols supporting cryptographic verification, provided
>>> +that the respective verification option is enabled. If unset, the user's
>>> +keyring is used.
>>>  .TP
>>>  .B sync-rsync-vcs-ignore = true|false
>>>  Ignore vcs directories that may be present in the repository. It is the
>>> diff --git a/pym/portage/repository/config.py 
>>> b/pym/portage/repository/config.py
>>> index be31ed3b1..d3a622f7c 100644
>>> --- a/pym/portage/repository/config.py
>>> +++ b/pym/portage/repository/config.py
>>> @@ -86,6 +86,7 @@ class RepoConfig(object):
>>> 'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 
>>> 'thin_manifest',
>>> 'update_changelog', '_eapis_banned', '_eapis_deprecated',
>>> '_masters_orig', 'module_specific_options', 
>>> 'manifest_required_hashes',
>>> +   'openpgp_key_path',
>>> )
>>>  
>>> def __init__(self, name, repo_opts, local_config=True):
>>> @@ -182,6 +183,9 @@ class RepoConfig(object):
>>> self.strict_misc_digests = repo_opts.get(
>>> 'strict-misc-digests', 'true').lower() == 'true'
>>>  
>>> +   self.openpgp_key_path = repo_opts.get(
>>> +   'sync-openpgp-key-path', None)
>>> +
>>> self.module_specific_options = {}
>>>  
>>> # Not implemented.
>>> diff --git a/pym/portage/sync/modules/rsync/__init__.py 
>>> b/pym/portage/sync/modules/rsync/__init__.py
>>> index 14af2120c..27a2548c0 100644
>>> --- a/pym/portage/sync/modules/rsync/__init__.py
>>> +++ b/pym/portage/sync/modules/rsync/__init__.py
>>> @@ -27,7 +27,6 @@ module_spec = {
>>> 'validate_config': CheckSyncConfig,
>>> 'module_specific_options': (
>>> 'sync-rsync-extra-opts',
>>> -   'sync-rsync-openpgp-key-path',
>>> 'sync-rsync-vcs-ignore',
>>> 'sync-rsync-verify-jobs',
>>> 'sync-rsync-verify-metamanifest',
>>> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
>>> b/pym/portage/sync/modules/rsync/rsync.py
>>> index 552ac6f6b..d9d7d56f2 100644
>>> --- a/pym/portage/sync/modules/rsync/rsync.py
>>> +++ b/pym/portage/sync/modules/rsync/rsync.py
>>> @@ -87,10 +87,6 @@ class RsyncSync(NewBase):
>>> self.verify_metamanifest = (
>>> self.repo.module_specific_options.get(
>>> 'sync-rsync-verify-metamanifest', 
>>> False))
>>> -   # Default to gentoo-keys keyring.
>>> -   self.openpgp_key_path = (
>>> -   self.repo.module_specific_options.get(
>>> -   'sync-rsync-openpgp-key-path', None))
>>> # Support overriding job count.
>>> self.verify_jobs = self.repo.module_specific_options.get(
>>> 'sync-rsync-verify-jobs', None)
>>> @@ -276,8 +272,8

Re: [gentoo-portage-dev] [PATCH v2] rsync: Fix *-verify-metamanifest boolean parsing

2018-01-30 Thread Zac Medico 
On 01/30/2018 11:18 AM, Michał Górny wrote:
> Fix sync-rsync-verify-metamanifest to correctly parse yes|no. Also
> correct the manpage to use those two terms as they were the ones used
> in repos.conf and the news item.
> ---
>  man/portage.5   | 4 ++--
>  pym/portage/sync/modules/rsync/rsync.py | 5 +++--
>  2 files changed, 5 insertions(+), 4 deletions(-)
> 
> v2: changed to support 'true' as well.
> 
> diff --git a/man/portage.5 b/man/portage.5
> index 1f6259715..d4f755f51 100644
> --- a/man/portage.5
> +++ b/man/portage.5
> @@ -1086,9 +1086,9 @@ directories if appropriate.
>  Number of parallel jobs to use when verifying nested Manifests. Defaults
>  to the apparent number of processors.
>  .TP
> -.B sync\-rsync\-verify\-metamanifest = true|false
> +.B sync\-rsync\-verify\-metamanifest = yes|no
>  Require the repository to contain a signed MetaManifest and verify
> -it using \fBapp\-portage/gemato\fR. Defaults to false.
> +it using \fBapp\-portage/gemato\fR. Defaults to no.
>  
>  .RE
>  
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index d9d7d56f2..7c020a563 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -83,10 +83,11 @@ class RsyncSync(NewBase):
>   
> self.repo.module_specific_options['sync-rsync-extra-opts']))
>  
>   # Process GLEP74 verification options.
> - # Default verification to 'on' for ::gentoo, 'off' otherwise.
> + # Default verification to 'no'; it's enabled for ::gentoo
> + # via default repos.conf though.
>   self.verify_metamanifest = (
>   self.repo.module_specific_options.get(
> - 'sync-rsync-verify-metamanifest', 
> False))
> + 'sync-rsync-verify-metamanifest', 'no') 
> in ('yes', 'true'))
>   # Support overriding job count.
>   self.verify_jobs = self.repo.module_specific_options.get(
>   'sync-rsync-verify-jobs', None)
> 

Looks good.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH v2] rsync: Fix *-verify-metamanifest boolean parsing

2018-01-30 Thread Michał Górny 
Fix sync-rsync-verify-metamanifest to correctly parse yes|no. Also
correct the manpage to use those two terms as they were the ones used
in repos.conf and the news item.
---
 man/portage.5   | 4 ++--
 pym/portage/sync/modules/rsync/rsync.py | 5 +++--
 2 files changed, 5 insertions(+), 4 deletions(-)

v2: changed to support 'true' as well.

diff --git a/man/portage.5 b/man/portage.5
index 1f6259715..d4f755f51 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1086,9 +1086,9 @@ directories if appropriate.
 Number of parallel jobs to use when verifying nested Manifests. Defaults
 to the apparent number of processors.
 .TP
-.B sync\-rsync\-verify\-metamanifest = true|false
+.B sync\-rsync\-verify\-metamanifest = yes|no
 Require the repository to contain a signed MetaManifest and verify
-it using \fBapp\-portage/gemato\fR. Defaults to false.
+it using \fBapp\-portage/gemato\fR. Defaults to no.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index d9d7d56f2..7c020a563 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -83,10 +83,11 @@ class RsyncSync(NewBase):

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
# Process GLEP74 verification options.
-   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   # Default verification to 'no'; it's enabled for ::gentoo
+   # via default repos.conf though.
self.verify_metamanifest = (
self.repo.module_specific_options.get(
-   'sync-rsync-verify-metamanifest', 
False))
+   'sync-rsync-verify-metamanifest', 'no') 
in ('yes', 'true'))
# Support overriding job count.
self.verify_jobs = self.repo.module_specific_options.get(
'sync-rsync-verify-jobs', None)
-- 
2.16.1

Re: [gentoo-portage-dev] [PATCH] rsync: Fix *-verify-metamanifest boolean parsing

2018-01-30 Thread Michał Górny 
W dniu wto, 30.01.2018 o godzinie 11∶07 -0800, użytkownik Zac Medico
napisał:
> On 01/30/2018 10:53 AM, Michał Górny wrote:
> > Fix sync-rsync-verify-metamanifest to correctly parse yes|no. Also
> > correct the manpage to use those two terms as they were the ones used
> > in repos.conf and the news item.
> > ---
> >  man/portage.5   | 4 ++--
> >  pym/portage/sync/modules/rsync/rsync.py | 5 +++--
> >  2 files changed, 5 insertions(+), 4 deletions(-)
> > 
> > diff --git a/man/portage.5 b/man/portage.5
> > index 1f6259715..d4f755f51 100644
> > --- a/man/portage.5
> > +++ b/man/portage.5
> > @@ -1086,9 +1086,9 @@ directories if appropriate.
> >  Number of parallel jobs to use when verifying nested Manifests. Defaults
> >  to the apparent number of processors.
> >  .TP
> > -.B sync\-rsync\-verify\-metamanifest = true|false
> > +.B sync\-rsync\-verify\-metamanifest = yes|no
> >  Require the repository to contain a signed MetaManifest and verify
> > -it using \fBapp\-portage/gemato\fR. Defaults to false.
> > +it using \fBapp\-portage/gemato\fR. Defaults to no.
> >  
> >  .RE
> >  
> > diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> > b/pym/portage/sync/modules/rsync/rsync.py
> > index d9d7d56f2..651e2d9f9 100644
> > --- a/pym/portage/sync/modules/rsync/rsync.py
> > +++ b/pym/portage/sync/modules/rsync/rsync.py
> > @@ -83,10 +83,11 @@ class RsyncSync(NewBase):
> > 
> > self.repo.module_specific_options['sync-rsync-extra-opts']))
> >  
> > # Process GLEP74 verification options.
> > -   # Default verification to 'on' for ::gentoo, 'off' otherwise.
> > +   # Default verification to 'no'; it's enabled for ::gentoo
> > +   # via default repos.conf though.
> > self.verify_metamanifest = (
> > self.repo.module_specific_options.get(
> > -   'sync-rsync-verify-metamanifest', 
> > False))
> > +   'sync-rsync-verify-metamanifest', 'no') 
> > == 'yes')
> 
> Many of our repos.conf options are currently using true|false, so this
> seems a lot more friendly:
> 
> self.repo.module_specific_options.get(
>   'sync-rsync-verify-metamanifest', 'no') in ('true', 'yes')
> 

I was actually thinking of introducing a bigger boolean parsing function
in a separate patch. But can do that for now.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] [PATCH] rsync: Fix *-verify-metamanifest boolean parsing

2018-01-30 Thread Zac Medico 
On 01/30/2018 10:53 AM, Michał Górny wrote:
> Fix sync-rsync-verify-metamanifest to correctly parse yes|no. Also
> correct the manpage to use those two terms as they were the ones used
> in repos.conf and the news item.
> ---
>  man/portage.5   | 4 ++--
>  pym/portage/sync/modules/rsync/rsync.py | 5 +++--
>  2 files changed, 5 insertions(+), 4 deletions(-)
> 
> diff --git a/man/portage.5 b/man/portage.5
> index 1f6259715..d4f755f51 100644
> --- a/man/portage.5
> +++ b/man/portage.5
> @@ -1086,9 +1086,9 @@ directories if appropriate.
>  Number of parallel jobs to use when verifying nested Manifests. Defaults
>  to the apparent number of processors.
>  .TP
> -.B sync\-rsync\-verify\-metamanifest = true|false
> +.B sync\-rsync\-verify\-metamanifest = yes|no
>  Require the repository to contain a signed MetaManifest and verify
> -it using \fBapp\-portage/gemato\fR. Defaults to false.
> +it using \fBapp\-portage/gemato\fR. Defaults to no.
>  
>  .RE
>  
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index d9d7d56f2..651e2d9f9 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -83,10 +83,11 @@ class RsyncSync(NewBase):
>   
> self.repo.module_specific_options['sync-rsync-extra-opts']))
>  
>   # Process GLEP74 verification options.
> - # Default verification to 'on' for ::gentoo, 'off' otherwise.
> + # Default verification to 'no'; it's enabled for ::gentoo
> + # via default repos.conf though.
>   self.verify_metamanifest = (
>   self.repo.module_specific_options.get(
> - 'sync-rsync-verify-metamanifest', 
> False))
> + 'sync-rsync-verify-metamanifest', 'no') 
> == 'yes')

Many of our repos.conf options are currently using true|false, so this
seems a lot more friendly:

self.repo.module_specific_options.get(
'sync-rsync-verify-metamanifest', 'no') in ('true', 'yes')

>   # Support overriding job count.
>   self.verify_jobs = self.repo.module_specific_options.get(
>   'sync-rsync-verify-jobs', None)
> 


-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH v4] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Michał Górny 
Warn about empty directories installed to /var in install-qa-check phase
(that were not "filled" using keepdir), to help developers stop relying
upon Portage preserving them. Those directories are rather unlikely to
be false positives.

Furthermore, remove all the empty directories if FEATURES=strict-keepdir
is used to catch even more problems (intended for developers). Here
warnings are not really suitable since there will be a high number
of false positives.

The PMS specifies the behavior upon merging empty directories
as undefined, and specifically prohibits ebuilds from attempting
to install empty directories. However, ebuilds occasionally still fall
into the trap of relying on 'dodir' preserving the directory. Make
the Portage behavior more strict in order to prevent that.

[v4: now with LC_COLLATE=C for sorting]
---
 bin/install-qa-check.d/95empty-dirs | 42 +
 man/make.conf.5 |  4 
 pym/portage/const.py|  1 +
 3 files changed, 47 insertions(+)
 create mode 100644 bin/install-qa-check.d/95empty-dirs

diff --git a/bin/install-qa-check.d/95empty-dirs 
b/bin/install-qa-check.d/95empty-dirs
new file mode 100644
index 0..8599db395
--- /dev/null
+++ b/bin/install-qa-check.d/95empty-dirs
@@ -0,0 +1,42 @@
+# Warn about and/or remove empty directories installed by ebuild.
+
+# Rationale: PMS prohibits ebuilds from installing empty directories.
+# Cleaning them up from the installation image provides an easy way
+# to make sure that ebuilds are not relying on it while making it easy
+# for users to override this if they need to.
+#
+# The ebuilds that need to preserve empty directories should use keepdir
+# as documented e.g.:
+# https://devmanual.gentoo.org/function-reference/install-functions/index.html
+#
+# For now, we emit QA warnings for empty directories in /var.
+# Additionally, if FEATURES=strict-keepdir is enabled we explicitly
+# remove *all* empty directories to trigger breakage.
+
+find_empty_dirs() {
+   local warn_dirs=()
+   local d striparg=
+
+   [[ ${FEATURES} == *strict-keepdir* ]] && striparg=-delete
+
+   while IFS= read -r -d $'\0' d; do
+   [[ ${d} == ${ED%/}/var/* ]] && warn_dirs+=( "${d}" )
+   done < <(find "${ED}" -depth -mindepth 1 -type d -empty -print0 
${striparg} | LC_COLLATE=C sort -z)
+
+   if [[ ${warn_dirs[@]} ]]; then
+   eqawarn "One or more empty directories installed to /var:"
+   eqawarn
+   for d in "${warn_dirs[@]}"; do
+   eqawarn "  ${d#${ED%/}}"
+   done
+   eqawarn
+   eqawarn "If those directories need to be preserved, please make 
sure to create"
+   eqawarn "or mark them for keeping using 'keepdir'. Future 
versions of Portage"
+   eqawarn "will strip empty directories from installation image."
+   fi
+}
+
+find_empty_dirs
+: # guarantee successful exit
+
+# vim:ft=sh
diff --git a/man/make.conf.5 b/man/make.conf.5
index a81b497bd..cb0f00237 100644
--- a/man/make.conf.5
+++ b/man/make.conf.5
@@ -623,6 +623,10 @@ see \fBinstallsources\fR.
 Have portage react strongly to conditions that have the potential to be
 dangerous (like missing or incorrect digests for ebuilds).
 .TP
+.B strict-keepdir
+Have portage strictly require keepdir calls in ebuilds.  Empty
+directories installed without explicit keepdir will be removed.
+.TP
 .B stricter
 Have portage react strongly to conditions that may conflict with system
 security provisions (for example textrels, executable stack).  Read about
diff --git a/pym/portage/const.py b/pym/portage/const.py
index e5fa4b67c..655be82b1 100644
--- a/pym/portage/const.py
+++ b/pym/portage/const.py
@@ -184,6 +184,7 @@ SUPPORTED_FEATURES   = frozenset([
"split-elog",
"split-log",
"strict",
+   "strict-keepdir",
"stricter",
"suidctl",
"test",
-- 
2.16.1

Re: [gentoo-portage-dev] [PATCH v3] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Michał Górny 
W dniu wto, 30.01.2018 o godzinie 10∶39 -0800, użytkownik Zac Medico
napisał:
> On 01/30/2018 10:18 AM, Zac Medico wrote:
> > On 01/29/2018 11:23 PM, Michał Górny wrote:
> > > Warn about empty directories installed to /var in install-qa-check phase
> > > (that were not "filled" using keepdir), to help developers stop relying
> > > upon Portage preserving them. Those directories are rather unlikely to
> > > be false positives.
> > > 
> > > Furthermore, remove all the empty directories if FEATURES=strict-keepdir
> > > is used to catch even more problems (intended for developers). Here
> > > warnings are not really suitable since there will be a high number
> > > of false positives.
> > > 
> > > The PMS specifies the behavior upon merging empty directories
> > > as undefined, and specifically prohibits ebuilds from attempting
> > > to install empty directories. However, ebuilds occasionally still fall
> > > into the trap of relying on 'dodir' preserving the directory. Make
> > > the Portage behavior more strict in order to prevent that.
> > > ---
> > >  bin/install-qa-check.d/95empty-dirs | 42 
> > > +
> > >  man/make.conf.5 |  4 
> > >  pym/portage/const.py|  1 +
> > >  3 files changed, 47 insertions(+)
> > >  create mode 100644 bin/install-qa-check.d/95empty-dirs
> > > 
> > > diff --git a/bin/install-qa-check.d/95empty-dirs 
> > > b/bin/install-qa-check.d/95empty-dirs
> > > new file mode 100644
> > > index 0..0d06b278d
> > > --- /dev/null
> > > +++ b/bin/install-qa-check.d/95empty-dirs
> > > @@ -0,0 +1,42 @@
> > > +# Warn about and/or remove empty directories installed by ebuild.
> > > +
> > > +# Rationale: PMS prohibits ebuilds from installing empty directories.
> > > +# Cleaning them up from the installation image provides an easy way
> > > +# to make sure that ebuilds are not relying on it while making it easy
> > > +# for users to override this if they need to.
> > > +#
> > > +# The ebuilds that need to preserve empty directories should use keepdir
> > > +# as documented e.g.:
> > > +# 
> > > https://devmanual.gentoo.org/function-reference/install-functions/index.html
> > > +#
> > > +# For now, we emit QA warnings for empty directories in /var.
> > > +# Additionally, if FEATURES=strict-keepdir is enabled we explicitly
> > > +# remove *all* empty directories to trigger breakage.
> > > +
> > > +find_empty_dirs() {
> > > + local warn_dirs=()
> > > + local d striparg=
> > > +
> > > + [[ ${FEATURES} == *strict-keepdir* ]] && striparg=-delete
> > > +
> > > + while IFS= read -r -d $'\0' d; do
> > > + [[ ${d} == ${ED%/}/var/* ]] && warn_dirs+=( "${d}" )
> > > + done < <(find "${ED}" -depth -mindepth 1 -type d -empty -print0 
> > > ${striparg} | sort -z)
> > 
> > Are you sure that this sort call is guaranteed to produce the correct
> > order? Comparison of '-' characters with '/' characters can lead to odd
> > results like this:
> > 
> > $ printf 'foo/bar\nfoo-bar/baz\nfoo/bar/baz\n' | sort
> > foo/bar
> > foo-bar/baz
> > foo/bar/baz
> 
> The sort is only for display purposes, maybe use LC_ALL=C for locale
> independence?

That's what I wanted to suggest. Well, LC_COLLATE=C should be enough.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] [PATCH] sync-rsync-openpgp-key-path -> generic sync-openpgp-key-path

2018-01-30 Thread Michał Górny 
W dniu wto, 30.01.2018 o godzinie 09∶32 -0800, użytkownik Zac Medico
napisał:
> On 01/30/2018 08:59 AM, Michał Górny wrote:
> > Rename the 'sync-rsync-openpgp-key-path' to a more generic
> > 'sync-openpgp-key-path'. OpenPGP is the basis of at least three
> > different verification schemes (git, rsync, snapshots) and at least
> > two of them use the same keys.
> > ---
> >  cnf/repos.conf | 2 +-
> >  man/portage.5  | 9 +
> >  pym/portage/repository/config.py   | 4 
> >  pym/portage/sync/modules/rsync/__init__.py | 1 -
> >  pym/portage/sync/modules/rsync/rsync.py| 8 ++--
> >  5 files changed, 12 insertions(+), 12 deletions(-)
> > 
> > diff --git a/cnf/repos.conf b/cnf/repos.conf
> > index 0d2b1f4be..4a40ff4fc 100644
> > --- a/cnf/repos.conf
> > +++ b/cnf/repos.conf
> > @@ -7,7 +7,7 @@ sync-type = rsync
> >  sync-uri = rsync://rsync.gentoo.org/gentoo-portage
> >  auto-sync = yes
> >  sync-rsync-verify-metamanifest = yes
> > -sync-rsync-openpgp-key-path = 
> > /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
> > +sync-openpgp-key-path = 
> > /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
> >  
> >  # for daily squashfs snapshots
> >  #sync-type = squashdelta
> > diff --git a/man/portage.5 b/man/portage.5
> > index 84999bd2f..1f6259715 100644
> > --- a/man/portage.5
> > +++ b/man/portage.5
> > @@ -1071,10 +1071,11 @@ Extra options to give to rsync on repository 
> > synchronization. It takes
> >  precedence over a declaration in [DEFAULT] section, that takes
> >  precedence over PORTAGE_RSYNC_EXTRA_OPTS.
> >  .TP
> > -.B sync\-rsync\-openpgp\-key\-path
> > -Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
> > -if \fBsync\-rsync\-verify\-metamanifest\fR is enabled. If unset,
> > -the user's keyring is used.
> > +.B sync\-openpgp\-key\-path
> > +Path to the OpenPGP key(ring) used to verify received repository. Used
> > +only for protocols supporting cryptographic verification, provided
> > +that the respective verification option is enabled. If unset, the user's
> > +keyring is used.
> >  .TP
> >  .B sync-rsync-vcs-ignore = true|false
> >  Ignore vcs directories that may be present in the repository. It is the
> > diff --git a/pym/portage/repository/config.py 
> > b/pym/portage/repository/config.py
> > index be31ed3b1..d3a622f7c 100644
> > --- a/pym/portage/repository/config.py
> > +++ b/pym/portage/repository/config.py
> > @@ -86,6 +86,7 @@ class RepoConfig(object):
> > 'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 
> > 'thin_manifest',
> > 'update_changelog', '_eapis_banned', '_eapis_deprecated',
> > '_masters_orig', 'module_specific_options', 
> > 'manifest_required_hashes',
> > +   'openpgp_key_path',
> > )
> >  
> > def __init__(self, name, repo_opts, local_config=True):
> > @@ -182,6 +183,9 @@ class RepoConfig(object):
> > self.strict_misc_digests = repo_opts.get(
> > 'strict-misc-digests', 'true').lower() == 'true'
> >  
> > +   self.openpgp_key_path = repo_opts.get(
> > +   'sync-openpgp-key-path', None)
> > +
> > self.module_specific_options = {}
> >  
> > # Not implemented.
> > diff --git a/pym/portage/sync/modules/rsync/__init__.py 
> > b/pym/portage/sync/modules/rsync/__init__.py
> > index 14af2120c..27a2548c0 100644
> > --- a/pym/portage/sync/modules/rsync/__init__.py
> > +++ b/pym/portage/sync/modules/rsync/__init__.py
> > @@ -27,7 +27,6 @@ module_spec = {
> > 'validate_config': CheckSyncConfig,
> > 'module_specific_options': (
> > 'sync-rsync-extra-opts',
> > -   'sync-rsync-openpgp-key-path',
> > 'sync-rsync-vcs-ignore',
> > 'sync-rsync-verify-jobs',
> > 'sync-rsync-verify-metamanifest',
> > diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> > b/pym/portage/sync/modules/rsync/rsync.py
> > index 552ac6f6b..d9d7d56f2 100644
> > --- a/pym/portage/sync/modules/rsync/rsync.py
> > +++ b/pym/portage/sync/modules/rsync/rsync.py
> > @@ -87,10 +87,6 @@ class RsyncSync(NewBase):
> > self.verify_metamanifest = (
> > self.repo.module_specific_options.get(
> > 'sync-rsync-verify-metamanifest', 
> > False))
> > -   # Default to gentoo-keys keyring.
> > -   self.openpgp_key_path = (
> > -   self.repo.module_specific_options.get(
> > -   'sync-rsync-openpgp-key-path', None))
> > # Support overriding job count.
> > self.verify_jobs = self.repo.module_specific_options.get(
> > 'sync-rsync-verify-jobs', None)
> > @@ -276,8 +272,8 @@ class RsyncSync(NewBase):
> > # if

[gentoo-portage-dev] [PATCH] rsync: Fix *-verify-metamanifest boolean parsing

2018-01-30 Thread Michał Górny 
Fix sync-rsync-verify-metamanifest to correctly parse yes|no. Also
correct the manpage to use those two terms as they were the ones used
in repos.conf and the news item.
---
 man/portage.5   | 4 ++--
 pym/portage/sync/modules/rsync/rsync.py | 5 +++--
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/man/portage.5 b/man/portage.5
index 1f6259715..d4f755f51 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1086,9 +1086,9 @@ directories if appropriate.
 Number of parallel jobs to use when verifying nested Manifests. Defaults
 to the apparent number of processors.
 .TP
-.B sync\-rsync\-verify\-metamanifest = true|false
+.B sync\-rsync\-verify\-metamanifest = yes|no
 Require the repository to contain a signed MetaManifest and verify
-it using \fBapp\-portage/gemato\fR. Defaults to false.
+it using \fBapp\-portage/gemato\fR. Defaults to no.
 
 .RE
 
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index d9d7d56f2..651e2d9f9 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -83,10 +83,11 @@ class RsyncSync(NewBase):

self.repo.module_specific_options['sync-rsync-extra-opts']))
 
# Process GLEP74 verification options.
-   # Default verification to 'on' for ::gentoo, 'off' otherwise.
+   # Default verification to 'no'; it's enabled for ::gentoo
+   # via default repos.conf though.
self.verify_metamanifest = (
self.repo.module_specific_options.get(
-   'sync-rsync-verify-metamanifest', 
False))
+   'sync-rsync-verify-metamanifest', 'no') 
== 'yes')
# Support overriding job count.
self.verify_jobs = self.repo.module_specific_options.get(
'sync-rsync-verify-jobs', None)
-- 
2.16.1

Re: [gentoo-portage-dev] [PATCH v3] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Zac Medico 
On 01/30/2018 10:18 AM, Zac Medico wrote:
> On 01/29/2018 11:23 PM, Michał Górny wrote:
>> Warn about empty directories installed to /var in install-qa-check phase
>> (that were not "filled" using keepdir), to help developers stop relying
>> upon Portage preserving them. Those directories are rather unlikely to
>> be false positives.
>>
>> Furthermore, remove all the empty directories if FEATURES=strict-keepdir
>> is used to catch even more problems (intended for developers). Here
>> warnings are not really suitable since there will be a high number
>> of false positives.
>>
>> The PMS specifies the behavior upon merging empty directories
>> as undefined, and specifically prohibits ebuilds from attempting
>> to install empty directories. However, ebuilds occasionally still fall
>> into the trap of relying on 'dodir' preserving the directory. Make
>> the Portage behavior more strict in order to prevent that.
>> ---
>>  bin/install-qa-check.d/95empty-dirs | 42 
>> +
>>  man/make.conf.5 |  4 
>>  pym/portage/const.py|  1 +
>>  3 files changed, 47 insertions(+)
>>  create mode 100644 bin/install-qa-check.d/95empty-dirs
>>
>> diff --git a/bin/install-qa-check.d/95empty-dirs 
>> b/bin/install-qa-check.d/95empty-dirs
>> new file mode 100644
>> index 0..0d06b278d
>> --- /dev/null
>> +++ b/bin/install-qa-check.d/95empty-dirs
>> @@ -0,0 +1,42 @@
>> +# Warn about and/or remove empty directories installed by ebuild.
>> +
>> +# Rationale: PMS prohibits ebuilds from installing empty directories.
>> +# Cleaning them up from the installation image provides an easy way
>> +# to make sure that ebuilds are not relying on it while making it easy
>> +# for users to override this if they need to.
>> +#
>> +# The ebuilds that need to preserve empty directories should use keepdir
>> +# as documented e.g.:
>> +# 
>> https://devmanual.gentoo.org/function-reference/install-functions/index.html
>> +#
>> +# For now, we emit QA warnings for empty directories in /var.
>> +# Additionally, if FEATURES=strict-keepdir is enabled we explicitly
>> +# remove *all* empty directories to trigger breakage.
>> +
>> +find_empty_dirs() {
>> +local warn_dirs=()
>> +local d striparg=
>> +
>> +[[ ${FEATURES} == *strict-keepdir* ]] && striparg=-delete
>> +
>> +while IFS= read -r -d $'\0' d; do
>> +[[ ${d} == ${ED%/}/var/* ]] && warn_dirs+=( "${d}" )
>> +done < <(find "${ED}" -depth -mindepth 1 -type d -empty -print0 
>> ${striparg} | sort -z)
> 
> Are you sure that this sort call is guaranteed to produce the correct
> order? Comparison of '-' characters with '/' characters can lead to odd
> results like this:
> 
> $ printf 'foo/bar\nfoo-bar/baz\nfoo/bar/baz\n' | sort
> foo/bar
> foo-bar/baz
> foo/bar/baz

The sort is only for display purposes, maybe use LC_ALL=C for locale
independence?
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH v3] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Zac Medico 
On 01/29/2018 11:23 PM, Michał Górny wrote:
> Warn about empty directories installed to /var in install-qa-check phase
> (that were not "filled" using keepdir), to help developers stop relying
> upon Portage preserving them. Those directories are rather unlikely to
> be false positives.
> 
> Furthermore, remove all the empty directories if FEATURES=strict-keepdir
> is used to catch even more problems (intended for developers). Here
> warnings are not really suitable since there will be a high number
> of false positives.
> 
> The PMS specifies the behavior upon merging empty directories
> as undefined, and specifically prohibits ebuilds from attempting
> to install empty directories. However, ebuilds occasionally still fall
> into the trap of relying on 'dodir' preserving the directory. Make
> the Portage behavior more strict in order to prevent that.
> ---
>  bin/install-qa-check.d/95empty-dirs | 42 
> +
>  man/make.conf.5 |  4 
>  pym/portage/const.py|  1 +
>  3 files changed, 47 insertions(+)
>  create mode 100644 bin/install-qa-check.d/95empty-dirs
> 
> diff --git a/bin/install-qa-check.d/95empty-dirs 
> b/bin/install-qa-check.d/95empty-dirs
> new file mode 100644
> index 0..0d06b278d
> --- /dev/null
> +++ b/bin/install-qa-check.d/95empty-dirs
> @@ -0,0 +1,42 @@
> +# Warn about and/or remove empty directories installed by ebuild.
> +
> +# Rationale: PMS prohibits ebuilds from installing empty directories.
> +# Cleaning them up from the installation image provides an easy way
> +# to make sure that ebuilds are not relying on it while making it easy
> +# for users to override this if they need to.
> +#
> +# The ebuilds that need to preserve empty directories should use keepdir
> +# as documented e.g.:
> +# 
> https://devmanual.gentoo.org/function-reference/install-functions/index.html
> +#
> +# For now, we emit QA warnings for empty directories in /var.
> +# Additionally, if FEATURES=strict-keepdir is enabled we explicitly
> +# remove *all* empty directories to trigger breakage.
> +
> +find_empty_dirs() {
> + local warn_dirs=()
> + local d striparg=
> +
> + [[ ${FEATURES} == *strict-keepdir* ]] && striparg=-delete
> +
> + while IFS= read -r -d $'\0' d; do
> + [[ ${d} == ${ED%/}/var/* ]] && warn_dirs+=( "${d}" )
> + done < <(find "${ED}" -depth -mindepth 1 -type d -empty -print0 
> ${striparg} | sort -z)

Are you sure that this sort call is guaranteed to produce the correct
order? Comparison of '-' characters with '/' characters can lead to odd
results like this:

$ printf 'foo/bar\nfoo-bar/baz\nfoo/bar/baz\n' | sort
foo/bar
foo-bar/baz
foo/bar/baz
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] sync-rsync-openpgp-key-path -> generic sync-openpgp-key-path

2018-01-30 Thread Zac Medico 
On 01/30/2018 08:59 AM, Michał Górny wrote:
> Rename the 'sync-rsync-openpgp-key-path' to a more generic
> 'sync-openpgp-key-path'. OpenPGP is the basis of at least three
> different verification schemes (git, rsync, snapshots) and at least
> two of them use the same keys.
> ---
>  cnf/repos.conf | 2 +-
>  man/portage.5  | 9 +
>  pym/portage/repository/config.py   | 4 
>  pym/portage/sync/modules/rsync/__init__.py | 1 -
>  pym/portage/sync/modules/rsync/rsync.py| 8 ++--
>  5 files changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/cnf/repos.conf b/cnf/repos.conf
> index 0d2b1f4be..4a40ff4fc 100644
> --- a/cnf/repos.conf
> +++ b/cnf/repos.conf
> @@ -7,7 +7,7 @@ sync-type = rsync
>  sync-uri = rsync://rsync.gentoo.org/gentoo-portage
>  auto-sync = yes
>  sync-rsync-verify-metamanifest = yes
> -sync-rsync-openpgp-key-path = 
> /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
> +sync-openpgp-key-path = 
> /var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
>  
>  # for daily squashfs snapshots
>  #sync-type = squashdelta
> diff --git a/man/portage.5 b/man/portage.5
> index 84999bd2f..1f6259715 100644
> --- a/man/portage.5
> +++ b/man/portage.5
> @@ -1071,10 +1071,11 @@ Extra options to give to rsync on repository 
> synchronization. It takes
>  precedence over a declaration in [DEFAULT] section, that takes
>  precedence over PORTAGE_RSYNC_EXTRA_OPTS.
>  .TP
> -.B sync\-rsync\-openpgp\-key\-path
> -Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
> -if \fBsync\-rsync\-verify\-metamanifest\fR is enabled. If unset,
> -the user's keyring is used.
> +.B sync\-openpgp\-key\-path
> +Path to the OpenPGP key(ring) used to verify received repository. Used
> +only for protocols supporting cryptographic verification, provided
> +that the respective verification option is enabled. If unset, the user's
> +keyring is used.
>  .TP
>  .B sync-rsync-vcs-ignore = true|false
>  Ignore vcs directories that may be present in the repository. It is the
> diff --git a/pym/portage/repository/config.py 
> b/pym/portage/repository/config.py
> index be31ed3b1..d3a622f7c 100644
> --- a/pym/portage/repository/config.py
> +++ b/pym/portage/repository/config.py
> @@ -86,6 +86,7 @@ class RepoConfig(object):
>   'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 
> 'thin_manifest',
>   'update_changelog', '_eapis_banned', '_eapis_deprecated',
>   '_masters_orig', 'module_specific_options', 
> 'manifest_required_hashes',
> + 'openpgp_key_path',
>   )
>  
>   def __init__(self, name, repo_opts, local_config=True):
> @@ -182,6 +183,9 @@ class RepoConfig(object):
>   self.strict_misc_digests = repo_opts.get(
>   'strict-misc-digests', 'true').lower() == 'true'
>  
> + self.openpgp_key_path = repo_opts.get(
> + 'sync-openpgp-key-path', None)
> +
>   self.module_specific_options = {}
>  
>   # Not implemented.
> diff --git a/pym/portage/sync/modules/rsync/__init__.py 
> b/pym/portage/sync/modules/rsync/__init__.py
> index 14af2120c..27a2548c0 100644
> --- a/pym/portage/sync/modules/rsync/__init__.py
> +++ b/pym/portage/sync/modules/rsync/__init__.py
> @@ -27,7 +27,6 @@ module_spec = {
>   'validate_config': CheckSyncConfig,
>   'module_specific_options': (
>   'sync-rsync-extra-opts',
> - 'sync-rsync-openpgp-key-path',
>   'sync-rsync-vcs-ignore',
>   'sync-rsync-verify-jobs',
>   'sync-rsync-verify-metamanifest',
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index 552ac6f6b..d9d7d56f2 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -87,10 +87,6 @@ class RsyncSync(NewBase):
>   self.verify_metamanifest = (
>   self.repo.module_specific_options.get(
>   'sync-rsync-verify-metamanifest', 
> False))
> - # Default to gentoo-keys keyring.
> - self.openpgp_key_path = (
> - self.repo.module_specific_options.get(
> - 'sync-rsync-openpgp-key-path', None))
>   # Support overriding job count.
>   self.verify_jobs = self.repo.module_specific_options.get(
>   'sync-rsync-verify-jobs', None)
> @@ -276,8 +272,8 @@ class RsyncSync(NewBase):
>   # if synced successfully, verify now
>   if exitcode == 0 and self.verify_metamanifest:
>   command = ['gemato', 'verify', '-s', self.repo.location]
> - if self.openpgp_key_path is

Re: [gentoo-portage-dev] [PATCH] rsync: Support overriding number of jobs for verification

2018-01-30 Thread Zac Medico 
On 01/30/2018 08:54 AM, Michał Górny wrote:
> Requested by Ulrich Müller.
> ---
>  man/portage.5  | 4 
>  pym/portage/sync/modules/rsync/__init__.py | 1 +
>  pym/portage/sync/modules/rsync/rsync.py| 5 +
>  3 files changed, 10 insertions(+)
> 
> diff --git a/man/portage.5 b/man/portage.5
> index 2d444a86f..84999bd2f 100644
> --- a/man/portage.5
> +++ b/man/portage.5
> @@ -1081,6 +1081,10 @@ Ignore vcs directories that may be present in the 
> repository. It is the
>  user's responsibility to set sync-rsync-extra-opts to protect vcs
>  directories if appropriate.
>  .TP
> +.B sync\-rsync\-verify\-jobs
> +Number of parallel jobs to use when verifying nested Manifests. Defaults
> +to the apparent number of processors.
> +.TP
>  .B sync\-rsync\-verify\-metamanifest = true|false
>  Require the repository to contain a signed MetaManifest and verify
>  it using \fBapp\-portage/gemato\fR. Defaults to false.
> diff --git a/pym/portage/sync/modules/rsync/__init__.py 
> b/pym/portage/sync/modules/rsync/__init__.py
> index df9a1995a..14af2120c 100644
> --- a/pym/portage/sync/modules/rsync/__init__.py
> +++ b/pym/portage/sync/modules/rsync/__init__.py
> @@ -29,6 +29,7 @@ module_spec = {
>   'sync-rsync-extra-opts',
>   'sync-rsync-openpgp-key-path',
>   'sync-rsync-vcs-ignore',
> + 'sync-rsync-verify-jobs',
>   'sync-rsync-verify-metamanifest',
>   ),
>   }
> diff --git a/pym/portage/sync/modules/rsync/rsync.py 
> b/pym/portage/sync/modules/rsync/rsync.py
> index 47f0e1ea3..552ac6f6b 100644
> --- a/pym/portage/sync/modules/rsync/rsync.py
> +++ b/pym/portage/sync/modules/rsync/rsync.py
> @@ -91,6 +91,9 @@ class RsyncSync(NewBase):
>   self.openpgp_key_path = (
>   self.repo.module_specific_options.get(
>   'sync-rsync-openpgp-key-path', None))
> + # Support overriding job count.
> + self.verify_jobs = self.repo.module_specific_options.get(
> + 'sync-rsync-verify-jobs', None)
>  
>   # Real local timestamp file.
>   self.servertimestampfile = os.path.join(
> @@ -275,6 +278,8 @@ class RsyncSync(NewBase):
>   command = ['gemato', 'verify', '-s', self.repo.location]
>   if self.openpgp_key_path is not None:
>   command += ['-K', self.openpgp_key_path]
> + if self.verify_jobs is not None:
> + command += ['-j', self.verify_jobs]
>   exitcode = portage.process.spawn(command, 
> **self.spawn_kwargs)
>  
>   return (exitcode, updatecache_flg)
> 

Looks good.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH] sync-rsync-openpgp-key-path -> generic sync-openpgp-key-path

2018-01-30 Thread Michał Górny 
Rename the 'sync-rsync-openpgp-key-path' to a more generic
'sync-openpgp-key-path'. OpenPGP is the basis of at least three
different verification schemes (git, rsync, snapshots) and at least
two of them use the same keys.
---
 cnf/repos.conf | 2 +-
 man/portage.5  | 9 +
 pym/portage/repository/config.py   | 4 
 pym/portage/sync/modules/rsync/__init__.py | 1 -
 pym/portage/sync/modules/rsync/rsync.py| 8 ++--
 5 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/cnf/repos.conf b/cnf/repos.conf
index 0d2b1f4be..4a40ff4fc 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -7,7 +7,7 @@ sync-type = rsync
 sync-uri = rsync://rsync.gentoo.org/gentoo-portage
 auto-sync = yes
 sync-rsync-verify-metamanifest = yes
-sync-rsync-openpgp-key-path = 
/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
+sync-openpgp-key-path = 
/var/lib/gentoo/gkeys/keyrings/gentoo/release/pubring.gpg
 
 # for daily squashfs snapshots
 #sync-type = squashdelta
diff --git a/man/portage.5 b/man/portage.5
index 84999bd2f..1f6259715 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1071,10 +1071,11 @@ Extra options to give to rsync on repository 
synchronization. It takes
 precedence over a declaration in [DEFAULT] section, that takes
 precedence over PORTAGE_RSYNC_EXTRA_OPTS.
 .TP
-.B sync\-rsync\-openpgp\-key\-path
-Path to the OpenPGP key(ring) used to verify MetaManifest. Used only
-if \fBsync\-rsync\-verify\-metamanifest\fR is enabled. If unset,
-the user's keyring is used.
+.B sync\-openpgp\-key\-path
+Path to the OpenPGP key(ring) used to verify received repository. Used
+only for protocols supporting cryptographic verification, provided
+that the respective verification option is enabled. If unset, the user's
+keyring is used.
 .TP
 .B sync-rsync-vcs-ignore = true|false
 Ignore vcs directories that may be present in the repository. It is the
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index be31ed3b1..d3a622f7c 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -86,6 +86,7 @@ class RepoConfig(object):
'sync_type', 'sync_umask', 'sync_uri', 'sync_user', 
'thin_manifest',
'update_changelog', '_eapis_banned', '_eapis_deprecated',
'_masters_orig', 'module_specific_options', 
'manifest_required_hashes',
+   'openpgp_key_path',
)
 
def __init__(self, name, repo_opts, local_config=True):
@@ -182,6 +183,9 @@ class RepoConfig(object):
self.strict_misc_digests = repo_opts.get(
'strict-misc-digests', 'true').lower() == 'true'
 
+   self.openpgp_key_path = repo_opts.get(
+   'sync-openpgp-key-path', None)
+
self.module_specific_options = {}
 
# Not implemented.
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index 14af2120c..27a2548c0 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -27,7 +27,6 @@ module_spec = {
'validate_config': CheckSyncConfig,
'module_specific_options': (
'sync-rsync-extra-opts',
-   'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
'sync-rsync-verify-jobs',
'sync-rsync-verify-metamanifest',
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index 552ac6f6b..d9d7d56f2 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -87,10 +87,6 @@ class RsyncSync(NewBase):
self.verify_metamanifest = (
self.repo.module_specific_options.get(
'sync-rsync-verify-metamanifest', 
False))
-   # Default to gentoo-keys keyring.
-   self.openpgp_key_path = (
-   self.repo.module_specific_options.get(
-   'sync-rsync-openpgp-key-path', None))
# Support overriding job count.
self.verify_jobs = self.repo.module_specific_options.get(
'sync-rsync-verify-jobs', None)
@@ -276,8 +272,8 @@ class RsyncSync(NewBase):
# if synced successfully, verify now
if exitcode == 0 and self.verify_metamanifest:
command = ['gemato', 'verify', '-s', self.repo.location]
-   if self.openpgp_key_path is not None:
-   command += ['-K', self.openpgp_key_path]
+   if self.repo.openpgp_key_path is not None:
+   command +=

[gentoo-portage-dev] [PATCH] rsync: Support overriding number of jobs for verification

2018-01-30 Thread Michał Górny 
Requested by Ulrich Müller.
---
 man/portage.5  | 4 
 pym/portage/sync/modules/rsync/__init__.py | 1 +
 pym/portage/sync/modules/rsync/rsync.py| 5 +
 3 files changed, 10 insertions(+)

diff --git a/man/portage.5 b/man/portage.5
index 2d444a86f..84999bd2f 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1081,6 +1081,10 @@ Ignore vcs directories that may be present in the 
repository. It is the
 user's responsibility to set sync-rsync-extra-opts to protect vcs
 directories if appropriate.
 .TP
+.B sync\-rsync\-verify\-jobs
+Number of parallel jobs to use when verifying nested Manifests. Defaults
+to the apparent number of processors.
+.TP
 .B sync\-rsync\-verify\-metamanifest = true|false
 Require the repository to contain a signed MetaManifest and verify
 it using \fBapp\-portage/gemato\fR. Defaults to false.
diff --git a/pym/portage/sync/modules/rsync/__init__.py 
b/pym/portage/sync/modules/rsync/__init__.py
index df9a1995a..14af2120c 100644
--- a/pym/portage/sync/modules/rsync/__init__.py
+++ b/pym/portage/sync/modules/rsync/__init__.py
@@ -29,6 +29,7 @@ module_spec = {
'sync-rsync-extra-opts',
'sync-rsync-openpgp-key-path',
'sync-rsync-vcs-ignore',
+   'sync-rsync-verify-jobs',
'sync-rsync-verify-metamanifest',
),
}
diff --git a/pym/portage/sync/modules/rsync/rsync.py 
b/pym/portage/sync/modules/rsync/rsync.py
index 47f0e1ea3..552ac6f6b 100644
--- a/pym/portage/sync/modules/rsync/rsync.py
+++ b/pym/portage/sync/modules/rsync/rsync.py
@@ -91,6 +91,9 @@ class RsyncSync(NewBase):
self.openpgp_key_path = (
self.repo.module_specific_options.get(
'sync-rsync-openpgp-key-path', None))
+   # Support overriding job count.
+   self.verify_jobs = self.repo.module_specific_options.get(
+   'sync-rsync-verify-jobs', None)
 
# Real local timestamp file.
self.servertimestampfile = os.path.join(
@@ -275,6 +278,8 @@ class RsyncSync(NewBase):
command = ['gemato', 'verify', '-s', self.repo.location]
if self.openpgp_key_path is not None:
command += ['-K', self.openpgp_key_path]
+   if self.verify_jobs is not None:
+   command += ['-j', self.verify_jobs]
exitcode = portage.process.spawn(command, 
**self.spawn_kwargs)
 
return (exitcode, updatecache_flg)
-- 
2.16.1

Re: [gentoo-portage-dev] [PATCH v3] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Michał Górny 
W dniu wto, 30.01.2018 o godzinie 11∶08 -0500, użytkownik Michael
Orlitzky napisał:
> On 01/30/2018 02:23 AM, Michał Górny wrote:
> > Warn about empty directories installed to /var
> 
> Why only warn about /var, considering that FEATURES=strict-keepdir will
> delete the others? People will probably assume that if their package
> throws no warnings, it's strict-keepdir-safe.
> 

Because non-/var stuff will have a lot of false positives, and people
will start keepdir-ing everything. Tools are there to help people, not
to think for them.

-- 
Best regards,
Michał Górny

Re: [gentoo-portage-dev] [PATCH v3] install-qa-check: New QA check/cleanup for empty directories

2018-01-30 Thread Michael Orlitzky 
On 01/30/2018 02:23 AM, Michał Górny wrote:
> Warn about empty directories installed to /var

Why only warn about /var, considering that FEATURES=strict-keepdir will
delete the others? People will probably assume that if their package
throws no warnings, it's strict-keepdir-safe.