[gentoo-dev] Re: Monthly x11@ project status for April 2018
Matt Turner posted on Sun, 01 Apr 2018 20:08:35 -0700 as excerpted: > My list of to-do items consists of: > > == Fix x11-base/xorg-server suid/systemd situation == > https://bugs.gentoo.org/635102 > > Under some circumstances (kernel modesetting driver + systemd, I think) > Xorg should be able to run without root privileges. We were shipping a > USE=suid option without anyone knowing or understanding its purpose. FWIW I understood it, but also knew it broke X for me back when I first tried it. However... > For >=x11-base/xorg-server-1.20 I plan to ship the xserver in a way that > allows systemd/elogind users with kernel modesetting drivers to run Xorg > without root privileges. I expect to push version 1.19.99.902 (1.20 RC2) > into the tree soon with something working for systemd. I would very much > appreciate an ebuild patch from any elogind user as well as non-systemd > testing to make sure I haven't broken anything like I did with > 1.19.99.901. I noticed the recent no-superuser X changes here (on ~amd64), and decided to try it again... And now (after undoing an old hack I had to manually set SUID here) I have X running as my normal user. Thanks! =:^) FWIW, systemd with modesetting (amdgpu), as you suspected. startx (no *dm at all merged). X starts on top of the vt1 login. xorg- server-1.19.99.901-r1 > == Update packages to depend on x11-base/xorg-proto == > https://bugs.gentoo.org/651286 > > The new x11-base/xorg-proto package combines nearly all (28 in fact) of > the x11-proto/* packages into one, with a very fast Meson build system. > It installs on my laptop in less time than it takes to ./configure one > of the individual x11-proto/ packages. I've kept empty versions of the > x11-proto/ packages to ease the transition. I noticed that I didn't need many of the protos any longer here too, and figured it was a recombining. Thanks for the confirmation. =:^) And thanks for the roadmap to what's ahead re X. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
[gentoo-dev] Monthly mips@ project status for April 2018
I'd like to start giving ~monthly updates on the status of mips@ in Gentoo. Recently I received a Loongson 3A system (quad-core 1.35GHz, 16GB RAM, AMD graphics) which is significantly faster and more stable than any other mips system I have. mips@ is currently assigned or cc'd on 29 bugs. This number is down from probably >50 at the beginning of March. My list of to-do items consists of: == Switch to stable profiles == https://github.com/gentoo/gentoo/pull/7672 While keeping mips unstable (KEYWORDS=~mips) I plan to switch the profiles from exp to stable so that no new breakage is added. To that end, I've been working to test and keyword packages until we can flip the switch without adding any breakage ourselves. See the github pull request for the current status. I'll periodically rebase the branch as I add more keywords to the main tree. My plan is to add stable 17.0 mips profiles when the keywording is sorted out and kill two birds with one stone. == Loongson 3A support == The Loongson 3A system I received requires a number of out of tree patches to the kernel, gcc, binutils, and glibc. I've seen their developers working to upstream kernel patches, but it's slow going and there are a lot of them. I haven't seen much work to upstream the other patches. I'm not sure how to reasonably support hardware requiring so many out-of-tree patches. == n64 (the ABI!) support == https://bugs.gentoo.org/show_bug.cgi?id=442548 https://bugs.gentoo.org/show_bug.cgi?id=513042 I want to support the tree main ABIs: Old 32 ("o32"), New 32 ("n32"), and New 64 ("n64"). n32 is equivalent to amd64's x32 ABI: 64-bit ABI but with 32-bit pointers. It's the default ABI we use for 64-bit MIPS. I'd like to offer n64 support as well, but gcc is not quite ready. At this point I have figured out the gcc build system enough to hack something together that works. I just need to... upstream something I guess? == n64 toolchain in n32 system == https://bugs.gentoo.org/show_bug.cgi?id=477956 n32 (and o32) only offer 31-bits of address space (2GB). That's not sufficient to link large libraries like webkit-gtk. For n32 systems, I'd like the toolchain binaries to be n64, to avoid these problems. That's going to be some amount of work on top of regular n64 support, but hopefully not too much. multilib/ABI_MIPS support already exists and works. == stages and installation media == https://bugs.gentoo.org/show_bug.cgi?id=150402 https://bugs.gentoo.org/show_bug.cgi?id=348647 With so many subarchitectures, ABIs, and two byte orderings providing stages and installation media has been a pain point. I'd like to automate as much of this as possible. I really need Kumba to build a new SGI CD, but well, that bug's been open since 2006. signature.asc Description: Digital signature
[gentoo-dev] Monthly x11@ project status for April 2018
I'd like to start giving ~monthly updates on the status of x11@ packages in Gentoo. I hope it gives some insight into the status of a rather important set of packages and maybe encourages others to lend a hand to a very understaffed project when possible. I expect future reports to be significantly shorter. x11@ currently maintains 291 packages. This number is down from 328 after the removal of 37 drivers for very old hardware in bug 606132. x11@ is currently assigned or cc'd on 222 bugs. This number is down from more than 412 in February 2015 (I reported this on #gentoo-desktop after closing out a bunch of bugs that day). I work on media-libs/mesa professionally for Intel, so I am involved with upstream for a number of the projects maintained by x11@ and know the right people to contact for the others. My strategy maintaining x11 packages is to keep a git ebuild in sync with upstream changes. When a release is made, everything should already be ready on the Gentoo side, and I'm able to just copy the git ebuild. This has enabled me to provide new versions of packages very quickly. I believe it has also helped reduce the time to stabilizing packages, since users often test and report bugs against the git ebuilds. My list of to-do items consists of: == Fix x11-base/xorg-server suid/systemd situation == https://bugs.gentoo.org/635102 Under some circumstances (kernel modesetting driver + systemd, I think) Xorg should be able to run without root privileges. We were shipping a USE=suid option without anyone knowing or understanding its purpose. For >=x11-base/xorg-server-1.20 I plan to ship the xserver in a way that allows systemd/elogind users with kernel modesetting drivers to run Xorg without root privileges. I expect to push version 1.19.99.902 (1.20 RC2) into the tree soon with something working for systemd. I would very much appreciate an ebuild patch from any elogind user as well as non-systemd testing to make sure I haven't broken anything like I did with 1.19.99.901. == Update packages to depend on x11-base/xorg-proto == https://bugs.gentoo.org/651286 The new x11-base/xorg-proto package combines nearly all (28 in fact) of the x11-proto/* packages into one, with a very fast Meson build system. It installs on my laptop in less time than it takes to ./configure one of the individual x11-proto/ packages. I've kept empty versions of the x11-proto/ packages to ease the transition. Once the last two architectures have stabilized it (arm@ and arm64@), we can begin transitioning to depending on x11-base/xorg-proto directly instead of the x11-proto/ packages. If there's a way to have repoman alert developers to deprecated dependencies in the same way we handle deprecated eclasses, I'd like to know about it. == Remove x11-proto/printproto and x11-libs/libXp == https://bugs.gentoo.org/649076 Support for the Xprint extension was removed from xorg-server-1.6.0 released 9 years ago, yet a handful of packages still claimed a dependence on the Xprint client library libXp. We've nearly removed all traces of it, and are waiting on the last few bugs to be closed out. == Convert media-libs/mesa ebuild to build with Meson == Mesa has grown a Meson build system over the last few months. I plan to ship media-libs/mesa-18.1 (Q2 2018) using Meson. hanetzer in #gentoo-desktop has been working on this. == Convert media-libs/xorg-server ebuild to build with Meson == The Xserver has also grown a Meson build system. I think it will probably be in reasonable shape by 1.20, but I don't want to delay it getting into the tree for that. 1.21 will probably be a long way off, but I expect to ship it with Meson, if not something before. No work has been done on this. == Add and require glvnd == https://bugs.gentoo.org/606924 https://github.com/NVIDIA/libglvnd OpenGL has historically been provided by a vendor's libGL.so. That's caused a number of headaches for distributions, since both x11-drivers/nvidia-drivers and media-libs/mesa would like to install /usr/lib/libGL.so. The GL Vendor-Neutral Dispatch library ("glvnd") defines a new common library interface, which dispatches to the underlying hardware driver. Mesa and the NVIDIA drivers both have support, but it needs to be wired up in Gentoo. Doing this would allow us to get rid of app-eselect/eselect-opengl and a whole class of build failures. No work has been done on this. I plan to do it after Mesa is converted to build with Meson. == Drop app-eselect/eselect-mesa == https://bugs.gentoo.org/576334 While app-eselect/eselect-opengl allows users to switch between OpenGL implementations (x11-drivers/nvidia-drivers vs media-libs/mesa), app-eselect/eselect-mesa allows users to switch between Mesa drivers for the same hardware. This made sense a few years ago, when there were both Gallium and "classic" DRI drivers for the same hardware, but today only the 10 year old i915/i945 integrated graphics still falls into this category. Frankly, both of the driver options
[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2018-04-01 23:59 UTC
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2018-04-01 23:59 UTC. Removals: app-pda/p3nfs 20180330-18:07 dilfridge ea6e2f40a16 app-text/searchmonkey 20180328-21:43 asturm12272bc5a86 dev-tex/qtexengine 20180328-21:43 asturme8057e08c55 dev-util/drone-ui 20180330-16:52 mrueg 1599a85cd68 media-video/abby 20180328-20:48 asturm75d1824f032 net-fs/torus 20180328-21:53 zmedico ca7746f52fa Additions: app-emacs/ghub 20180331-06:22 graaff1eb09752210 app-metrics/prometheus-mysqld_exporter 20180326-22:38 whissi101f87f123c dev-go/fuzzy 20180328-21:49 mgorny6360f43aa6d dev-go/godebug-pretty 20180328-21:47 mgornye223e4b86ea dev-lang/mono-basic20180325-12:42 cynedecb1aff090b1 media-fonts/fira-code 20180331-07:39 graaff1bc69a5e200 media-fonts/iosevka20180326-14:57 mgorny218d847e9f3 net-analyzer/kapacitor 20180326-02:54 williamh f3ed20bf98a net-libs/srt 20180322-16:45 lu_zero aeccfd18102 net-misc/casync20180326-14:45 mrueg eb6b676bea2 sys-apps/smcipmitool 20180329-14:40 mgornyffa696581eb sys-cluster/pmix 20180401-20:43 jlec 69972533a7a sys-cluster/ucx20180401-20:36 jlec 978c8e081d9 sys-fs/compsize20180327-19:34 mgorny6b6d7b9052d sys-libs/rpmatch-standalone20180329-11:28 blueness 1ffea40c4a4 www-apps/hugo 20180330-15:18 mrueg b393e4812cd x11-misc/xflux 20180327-20:41 mgornyf8368de4a58 x11-misc/xflux-gui 20180329-12:56 mgorny133aec6aa2e -- Robin Hugh Johnson Gentoo Linux Developer E-Mail : robb...@gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 Removed Packages: app-pda/p3nfs,removed,dilfridge,20180330-18:07,ea6e2f40a16 dev-util/drone-ui,removed,mrueg,20180330-16:52,1599a85cd68 net-fs/torus,removed,zmedico,20180328-21:53,ca7746f52fa dev-tex/qtexengine,removed,asturm,20180328-21:43,e8057e08c55 app-text/searchmonkey,removed,asturm,20180328-21:43,12272bc5a86 media-video/abby,removed,asturm,20180328-20:48,75d1824f032 Added Packages: sys-cluster/pmix,added,jlec,20180401-20:43,69972533a7a sys-cluster/ucx,added,jlec,20180401-20:36,978c8e081d9 media-fonts/fira-code,added,graaff,20180331-07:39,1bc69a5e200 app-emacs/ghub,added,graaff,20180331-06:22,1eb09752210 sys-fs/compsize,added,mgorny,20180327-19:34,6b6d7b9052d dev-go/fuzzy,added,mgorny,20180328-21:49,6360f43aa6d dev-go/godebug-pretty,added,mgorny,20180328-21:47,e223e4b86ea www-apps/hugo,added,mrueg,20180330-15:18,b393e4812cd sys-apps/smcipmitool,added,mgorny,20180329-14:40,ffa696581eb x11-misc/xflux-gui,added,mgorny,20180329-12:56,133aec6aa2e sys-libs/rpmatch-standalone,added,blueness,20180329-11:28,1ffea40c4a4 x11-misc/xflux,added,mgorny,20180327-20:41,f8368de4a58 media-fonts/iosevka,added,mgorny,20180326-14:57,218d847e9f3 app-metrics/prometheus-mysqld_exporter,added,whissi,20180326-22:38,101f87f123c net-libs/srt,added,lu_zero,20180322-16:45,aeccfd18102 net-misc/casync,added,mrueg,20180326-14:45,eb6b676bea2 dev-lang/mono-basic,added,cynede,20180325-12:42,cb1aff090b1 net-analyzer/kapacitor,added,williamh,20180326-02:54,f3ed20bf98a Done.
[gentoo-dev] Packages up for grabs: app-office/mdbtools
Dear all, The following packages are up for grabs: app-office/mdbtools after retirement of the proxied maintainer. https://packages.gentoo.org/packages/app-office/mdbtools "Set of libraries and utilities for reading Microsoft Access database (MDB) files" Todo: HOMEPAGE links to SF, but we distribute via SRC_URI from github. It is unclear, if the project moved, or forked. See also: https://github.com/brianb/mdbtools/issues/14 CC to the former maintainer in the hope, that he can explain why this source was chosen. -- Best, Jonas signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)
On Sun, Apr 1, 2018 at 2:29 PM, Michał Górnywrote: > W dniu nie, 01.04.2018 o godzinie 08∶59 -0700, użytkownik Zac Medico > napisał: > > On 04/01/2018 03:57 AM, Michał Górny wrote: > > > W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico > > > napisał: > > > > Since key refresh is prone to failure, retry using exponential > > > > backoff with random jitter. This adds the following sync-openpgp-* > > > > configuration settings: > > > > > > > > sync-openpgp-key-refresh-retry-count = 40 > > > > > > > > Maximum number of times to retry key refresh if it fails. Between > > > > each key refresh attempt, there is an exponential delay with a > > > > constant multiplier and a uniform random multiplier between 0 and > 1. > > > > > > > > sync-openpgp-key-refresh-retry-delay-exp-base = 2 > > > > > > > > The base of the exponential expression. The exponent is the number > > > > of previous refresh attempts. > > > > > > > > sync-openpgp-key-refresh-retry-delay-max = 60 > > > > > > > > Maximum delay between each retry attempt, in units of seconds. > This > > > > places a limit on the length of the exponential delay. > > > > > > > > sync-openpgp-key-refresh-retry-delay-mult = 4 > > > > > > > > Multiplier for the exponential delay. > > > > > > > > sync-openpgp-key-refresh-retry-overall-timeout = 1200 > > > > > > > > Combined time limit for all refresh attempts, in units of seconds. > > > > > > > > Bug: https://bugs.gentoo.org/649276 > > > > > > > > Zac Medico (4): > > > > Add ForkExecutor (bug 649588) > > > > Add ExponentialBackoff and RandomExponentialBackoff > > > > Add retry decorator (API inspired by tenacity) > > > > rsync: add key refresh retry (bug 649276) > > > > > > > > cnf/repos.conf| 5 + > > > > man/portage.5 | 19 +++ > > > > pym/portage/repository/config.py | 22 > > > > pym/portage/sync/modules/rsync/rsync.py | 16 ++- > > > > pym/portage/sync/syncbase.py | 85 +++- > > > > pym/portage/tests/util/futures/test_retry.py | 147 > + > > > > pym/portage/util/_eventloop/EventLoop.py | 45 ++- > > > > pym/portage/util/backoff.py | 48 +++ > > > > pym/portage/util/futures/executor/__init__.py | 0 > > > > pym/portage/util/futures/executor/fork.py | 130 > +++ > > > > pym/portage/util/futures/futures.py | 6 + > > > > pym/portage/util/futures/retry.py | 178 > ++ > > > > 12 files changed, 697 insertions(+), 4 deletions(-) > > > > create mode 100644 pym/portage/tests/util/futures/test_retry.py > > > > create mode 100644 pym/portage/util/backoff.py > > > > create mode 100644 pym/portage/util/futures/executor/__init__.py > > > > create mode 100644 pym/portage/util/futures/executor/fork.py > > > > create mode 100644 pym/portage/util/futures/retry.py > > > > > > > > > > This essentially looks like ~700 lines of code to try to workaround > > > broken networking. I would rather try to do that using 5 lines of code > > > but that's just me, and my programs aren't enterprise quality. I just > > > hope it actually solves as many problems as it's going to introduce. > > > > The vast majority of this code is generic and reusable, and I do intend > > to reuse it. For example, the executor support will be an essential > > piece for the asyncio.AbstractEventLoop for bug 649588. > > Sure it is and sure you will. But tell me: who is going to maintain it > all? Because as far as I can see, we're still dealing with a bus factor > of one and all you're doing is making it worse. More code, more > complexity, more creeping featurism and hacks. > I'll split this reply into two sorts of points. One is a brief point about this patchset. The other is a larger point about the project and its direction. About this patchset: I don't mind this code because its not domain specific and I can read it and understand it. Many apps need async ways to run code, and anyone who has used the futures API will find this code familiar. I reviewed it in about 10 minutes. I'm not heavily invested with the async-wagon that Zac is driving, but I also see how its an area where performance can be improved by doing more stuff at once using an async operations. I think if you have more generic concerns with the async frameworks I'd love to see some discussion about them (particularly around how we can improve perf by using tighter algorithms; often the 'perf' boost is just higher CPU utilization and driving multi-core systems which results in better walltime perf, but not necessarily cpu-perf.) Its fine to disagree here. I think the project has some requirements around minimized dependencies, so the choice is either to pull in some kind of retry-lib and add a dep, or add this code. Its also not immediately clear if the 3rd party deps would still not
Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)
On 04/01/2018 11:29 AM, Michał Górny wrote: > W dniu nie, 01.04.2018 o godzinie 08∶59 -0700, użytkownik Zac Medico > napisał: >> On 04/01/2018 03:57 AM, Michał Górny wrote: >>> W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico >>> napisał: Since key refresh is prone to failure, retry using exponential backoff with random jitter. This adds the following sync-openpgp-* configuration settings: sync-openpgp-key-refresh-retry-count = 40 Maximum number of times to retry key refresh if it fails. Between each key refresh attempt, there is an exponential delay with a constant multiplier and a uniform random multiplier between 0 and 1. sync-openpgp-key-refresh-retry-delay-exp-base = 2 The base of the exponential expression. The exponent is the number of previous refresh attempts. sync-openpgp-key-refresh-retry-delay-max = 60 Maximum delay between each retry attempt, in units of seconds. This places a limit on the length of the exponential delay. sync-openpgp-key-refresh-retry-delay-mult = 4 Multiplier for the exponential delay. sync-openpgp-key-refresh-retry-overall-timeout = 1200 Combined time limit for all refresh attempts, in units of seconds. Bug: https://bugs.gentoo.org/649276 Zac Medico (4): Add ForkExecutor (bug 649588) Add ExponentialBackoff and RandomExponentialBackoff Add retry decorator (API inspired by tenacity) rsync: add key refresh retry (bug 649276) cnf/repos.conf| 5 + man/portage.5 | 19 +++ pym/portage/repository/config.py | 22 pym/portage/sync/modules/rsync/rsync.py | 16 ++- pym/portage/sync/syncbase.py | 85 +++- pym/portage/tests/util/futures/test_retry.py | 147 + pym/portage/util/_eventloop/EventLoop.py | 45 ++- pym/portage/util/backoff.py | 48 +++ pym/portage/util/futures/executor/__init__.py | 0 pym/portage/util/futures/executor/fork.py | 130 +++ pym/portage/util/futures/futures.py | 6 + pym/portage/util/futures/retry.py | 178 ++ 12 files changed, 697 insertions(+), 4 deletions(-) create mode 100644 pym/portage/tests/util/futures/test_retry.py create mode 100644 pym/portage/util/backoff.py create mode 100644 pym/portage/util/futures/executor/__init__.py create mode 100644 pym/portage/util/futures/executor/fork.py create mode 100644 pym/portage/util/futures/retry.py >>> >>> This essentially looks like ~700 lines of code to try to workaround >>> broken networking. I would rather try to do that using 5 lines of code >>> but that's just me, and my programs aren't enterprise quality. I just >>> hope it actually solves as many problems as it's going to introduce. >> >> The vast majority of this code is generic and reusable, and I do intend >> to reuse it. For example, the executor support will be an essential >> piece for the asyncio.AbstractEventLoop for bug 649588. > > Sure it is and sure you will. But tell me: who is going to maintain it > all? Because as far as I can see, we're still dealing with a bus factor > of one and all you're doing is making it worse. More code, more > complexity, more creeping featurism and hacks. In the worst case, people can disable the retry feature and all of the associated code can be disabled simply by setting sync-openpgp-key-refresh-retry-count = 0 in repos.conf. In order to reduce the bus factor, I'm modeling the code on the standard library's asyncio framework. This allows use to leverage people's experience with asyncio, and also introduces people to asyncio concepts if they are not yet familar. > Last time you went away, you left us with a horribly unmaintainable > package manager full of complexity, hacks and creeping featurism > and a Portage team whose members had barely any knowledge of the code. > Just when things started moving again, you came back and we're back to > square one. I'd love to help reduce the bus factor, and I'm working to do that, like with the asyncio stuff. > Today Portage once again is a one-developer project, full of more > complexity, more hacks and more creeping featurism. And we once again > have a bus factor of one -- one developer who apparently knows > everything, does everything and tries to be nice to everyone, except he > really ignores others, makes a lot of empty promises and consistently > makes the health of the project go from bad to worse. I disagree with your synopsis. > So, please tell me: what happens when you leave again? How have you used > your time in the project? What have you done to make sure that > the project stays
Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)
On 01-04-2018 20:29:59 +0200, Michał Górny wrote: > > > This essentially looks like ~700 lines of code to try to workaround > > > broken networking. I would rather try to do that using 5 lines of code > > > but that's just me, and my programs aren't enterprise quality. I just > > > hope it actually solves as many problems as it's going to introduce. > > > > The vast majority of this code is generic and reusable, and I do intend > > to reuse it. For example, the executor support will be an essential > > piece for the asyncio.AbstractEventLoop for bug 649588. > > Sure it is and sure you will. But tell me: who is going to maintain it > all? Because as far as I can see, we're still dealing with a bus factor > of one and all you're doing is making it worse. More code, more > complexity, more creeping featurism and hacks. Well, well, well. This could be said about a lot of code, including your own. > Last time you went away, you left us with a horribly unmaintainable > package manager full of complexity, hacks and creeping featurism > and a Portage team whose members had barely any knowledge of the code. > Just when things started moving again, you came back and we're back to > square one. I don't see why this has to be made personal. In the olde days people just pushed whatever they needed, now it's reviewed and acked, so how can it be back to square one? > Today Portage once again is a one-developer project, full of more > complexity, more hacks and more creeping featurism. And we once again > have a bus factor of one -- one developer who apparently knows > everything, does everything and tries to be nice to everyone, except he > really ignores others, makes a lot of empty promises and consistently > makes the health of the project go from bad to worse. Errr, didn't you recently start your own fork with creeping featurism of its own because mainline was/is too slow? > So, please tell me: what happens when you leave again? How have you used > your time in the project? What have you done to make sure that > the project stays alive without you? Because as far as I can see, adding > few thousand of lines of practically unreviewed code every month does > not help with that. Why is this Zac's problem specifically? Isn't this a general problem of Gentoo? And isn't your attitude contributing in a large factor to the bus-factor getting down from 1 to 0? > I forked Portage because I didn't want to fight with you. When I forked > it, I declared that I will merge mainline changes regularly for > the benefit of my users. But after a week, I really start feeling like > that's going to be a really bad idea. Like it's time to forget about > mainline Portage as a completely dead end, and go our separate ways. So you fork. Like many forks, that is because people feel that it isn't going "their way". Yay. That doesn't make you "right". > I'm seriously worried about the future of Gentoo. I'd really appreciate > if you started focusing on that as well. I get that all this stuff looks > cool on paper but few months or years from now, someone will curse > 'whoever wrote that code' while trying to fix some nasty bug. Or get > things moving forward. Or implement EAPI 8. Perhaps it's time to look into the mirror. Thanks, Fabian -- Fabian Groffen Gentoo on a different level signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)
W dniu nie, 01.04.2018 o godzinie 08∶59 -0700, użytkownik Zac Medico napisał: > On 04/01/2018 03:57 AM, Michał Górny wrote: > > W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico > > napisał: > > > Since key refresh is prone to failure, retry using exponential > > > backoff with random jitter. This adds the following sync-openpgp-* > > > configuration settings: > > > > > > sync-openpgp-key-refresh-retry-count = 40 > > > > > > Maximum number of times to retry key refresh if it fails. Between > > > each key refresh attempt, there is an exponential delay with a > > > constant multiplier and a uniform random multiplier between 0 and 1. > > > > > > sync-openpgp-key-refresh-retry-delay-exp-base = 2 > > > > > > The base of the exponential expression. The exponent is the number > > > of previous refresh attempts. > > > > > > sync-openpgp-key-refresh-retry-delay-max = 60 > > > > > > Maximum delay between each retry attempt, in units of seconds. This > > > places a limit on the length of the exponential delay. > > > > > > sync-openpgp-key-refresh-retry-delay-mult = 4 > > > > > > Multiplier for the exponential delay. > > > > > > sync-openpgp-key-refresh-retry-overall-timeout = 1200 > > > > > > Combined time limit for all refresh attempts, in units of seconds. > > > > > > Bug: https://bugs.gentoo.org/649276 > > > > > > Zac Medico (4): > > > Add ForkExecutor (bug 649588) > > > Add ExponentialBackoff and RandomExponentialBackoff > > > Add retry decorator (API inspired by tenacity) > > > rsync: add key refresh retry (bug 649276) > > > > > > cnf/repos.conf| 5 + > > > man/portage.5 | 19 +++ > > > pym/portage/repository/config.py | 22 > > > pym/portage/sync/modules/rsync/rsync.py | 16 ++- > > > pym/portage/sync/syncbase.py | 85 +++- > > > pym/portage/tests/util/futures/test_retry.py | 147 + > > > pym/portage/util/_eventloop/EventLoop.py | 45 ++- > > > pym/portage/util/backoff.py | 48 +++ > > > pym/portage/util/futures/executor/__init__.py | 0 > > > pym/portage/util/futures/executor/fork.py | 130 +++ > > > pym/portage/util/futures/futures.py | 6 + > > > pym/portage/util/futures/retry.py | 178 > > > ++ > > > 12 files changed, 697 insertions(+), 4 deletions(-) > > > create mode 100644 pym/portage/tests/util/futures/test_retry.py > > > create mode 100644 pym/portage/util/backoff.py > > > create mode 100644 pym/portage/util/futures/executor/__init__.py > > > create mode 100644 pym/portage/util/futures/executor/fork.py > > > create mode 100644 pym/portage/util/futures/retry.py > > > > > > > This essentially looks like ~700 lines of code to try to workaround > > broken networking. I would rather try to do that using 5 lines of code > > but that's just me, and my programs aren't enterprise quality. I just > > hope it actually solves as many problems as it's going to introduce. > > The vast majority of this code is generic and reusable, and I do intend > to reuse it. For example, the executor support will be an essential > piece for the asyncio.AbstractEventLoop for bug 649588. Sure it is and sure you will. But tell me: who is going to maintain it all? Because as far as I can see, we're still dealing with a bus factor of one and all you're doing is making it worse. More code, more complexity, more creeping featurism and hacks. Last time you went away, you left us with a horribly unmaintainable package manager full of complexity, hacks and creeping featurism and a Portage team whose members had barely any knowledge of the code. Just when things started moving again, you came back and we're back to square one. Today Portage once again is a one-developer project, full of more complexity, more hacks and more creeping featurism. And we once again have a bus factor of one -- one developer who apparently knows everything, does everything and tries to be nice to everyone, except he really ignores others, makes a lot of empty promises and consistently makes the health of the project go from bad to worse. So, please tell me: what happens when you leave again? How have you used your time in the project? What have you done to make sure that the project stays alive without you? Because as far as I can see, adding few thousand of lines of practically unreviewed code every month does not help with that. I forked Portage because I didn't want to fight with you. When I forked it, I declared that I will merge mainline changes regularly for the benefit of my users. But after a week, I really start feeling like that's going to be a really bad idea. Like it's time to forget about mainline Portage as a completely dead end, and go our separate ways. I'm seriously worried about the future of Gentoo. I'd really appreciate if you
Re: [gentoo-portage-dev] [PATCH] INSTALL_MASK: honor install time config for binary packages (bug 651952)
On 04/01/2018 06:54 AM, Michał Górny wrote: > W dniu czw, 29.03.2018 o godzinie 15∶34 -0700, użytkownik Zac Medico > napisał: >> For binary packages, honor the INSTALL_MASK configuration that >> exists at install time, since it might differ from the build time >> setting. >> >> Fixes: 3416876c0ee7 ("{,PKG_}INSTALL_MASK: python implementation") >> Bug: https://bugs.gentoo.org/651952 >> --- >> bin/misc-functions.sh| 23 +++ >> bin/phase-functions.sh | 10 +- >> pym/portage/dbapi/vartree.py | 5 + >> 3 files changed, 29 insertions(+), 9 deletions(-) >> >> diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh >> index 26f589915..a6330ee93 100755 >> --- a/bin/misc-functions.sh >> +++ b/bin/misc-functions.sh >> @@ -323,6 +323,29 @@ postinst_qa_check() { >> done < <(printf "%s\0" "${qa_checks[@]}" | LC_ALL=C sort -u -z) >> } >> >> +preinst_mask() { >> +# Remove man pages, info pages, docs if requested. This is >> +# implemented in bash in order to respect INSTALL_MASK settings >> +# from bashrc. >> +local f x >> +for f in man info doc; do >> +if has no${f} ${FEATURES}; then >> +INSTALL_MASK+=" /usr/share/${f}" >> +fi >> +done >> + >> +# Store modified variables in build-info. >> +cd "${PORTAGE_BUILDDIR}"/build-info || die >> +set -f >> + >> +IFS=$' \t\n\r' >> +for f in INSTALL_MASK; do > > This loop along with the whole indirection is entirely pointless, given > that you're processing exactly one variable. I did this for consistency with the related loop in dyn_install, since we might save values of other variables from the binary package environment. In fact, I think we should record DOC_SYMLINKS_DIR here, since it affects CONTENTS, much like INSTALL_MASK. >> +x=$(echo -n ${!f}) >> +[[ -n ${x} ]] && echo "${x}" > "${f}" > > There's probably no point in this [[ -n ... ]], as that: > > a. requires you to special-handle missing INSTALL_MASK file, while it's > easier to just ensure that it's there (and I think you requested > the same thing from me before you rewritten my commit into breakage), > > b. makes it impossible to distinguish packages from before INSTALL_MASK > storing was added from those where it is empty. Maybe this only matters within the context of bug 364633 [1], and for that I think we need to introduce a separate CONTENTS.INSTALL_MASK file so that we can easily toggle collision-protect behavior to use CONTENTS.INSTALL_MASK when desired. [1] https://bugs.gentoo.org/364633 -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)
On 04/01/2018 03:57 AM, Michał Górny wrote: > W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico > napisał: >> Since key refresh is prone to failure, retry using exponential >> backoff with random jitter. This adds the following sync-openpgp-* >> configuration settings: >> >> sync-openpgp-key-refresh-retry-count = 40 >> >> Maximum number of times to retry key refresh if it fails. Between >> each key refresh attempt, there is an exponential delay with a >> constant multiplier and a uniform random multiplier between 0 and 1. >> >> sync-openpgp-key-refresh-retry-delay-exp-base = 2 >> >> The base of the exponential expression. The exponent is the number >> of previous refresh attempts. >> >> sync-openpgp-key-refresh-retry-delay-max = 60 >> >> Maximum delay between each retry attempt, in units of seconds. This >> places a limit on the length of the exponential delay. >> >> sync-openpgp-key-refresh-retry-delay-mult = 4 >> >> Multiplier for the exponential delay. >> >> sync-openpgp-key-refresh-retry-overall-timeout = 1200 >> >> Combined time limit for all refresh attempts, in units of seconds. >> >> Bug: https://bugs.gentoo.org/649276 >> >> Zac Medico (4): >> Add ForkExecutor (bug 649588) >> Add ExponentialBackoff and RandomExponentialBackoff >> Add retry decorator (API inspired by tenacity) >> rsync: add key refresh retry (bug 649276) >> >> cnf/repos.conf| 5 + >> man/portage.5 | 19 +++ >> pym/portage/repository/config.py | 22 >> pym/portage/sync/modules/rsync/rsync.py | 16 ++- >> pym/portage/sync/syncbase.py | 85 +++- >> pym/portage/tests/util/futures/test_retry.py | 147 + >> pym/portage/util/_eventloop/EventLoop.py | 45 ++- >> pym/portage/util/backoff.py | 48 +++ >> pym/portage/util/futures/executor/__init__.py | 0 >> pym/portage/util/futures/executor/fork.py | 130 +++ >> pym/portage/util/futures/futures.py | 6 + >> pym/portage/util/futures/retry.py | 178 >> ++ >> 12 files changed, 697 insertions(+), 4 deletions(-) >> create mode 100644 pym/portage/tests/util/futures/test_retry.py >> create mode 100644 pym/portage/util/backoff.py >> create mode 100644 pym/portage/util/futures/executor/__init__.py >> create mode 100644 pym/portage/util/futures/executor/fork.py >> create mode 100644 pym/portage/util/futures/retry.py >> > > This essentially looks like ~700 lines of code to try to workaround > broken networking. I would rather try to do that using 5 lines of code > but that's just me, and my programs aren't enterprise quality. I just > hope it actually solves as many problems as it's going to introduce. The vast majority of this code is generic and reusable, and I do intend to reuse it. For example, the executor support will be an essential piece for the asyncio.AbstractEventLoop for bug 649588. -- Thanks, Zac signature.asc Description: OpenPGP digital signature
Re: [gentoo-portage-dev] [PATCH 3/4] Add retry decorator (API inspired by tenacity)
also lgtm. On Sat, Mar 31, 2018 at 10:46 PM, Zac Medicowrote: > This decorator will be useful for retrying asynchronous > operations, such as gpg key refresh (bug 649276). The > API is inspired by tenacity, but is simpler. Only > asynchronous functions (like @asyncio.coroutine functions) > are supported. In order to retry a synchronous function, > first convert it to an asynchronous function as follows: > > asynchronous_func = functools.partial( > loop.run_in_executor, None, synchronous_func) > > Bug: https://bugs.gentoo.org/649276 > See: https://github.com/jd/tenacity > --- > pym/portage/tests/util/futures/test_retry.py | 147 ++ > pym/portage/util/futures/futures.py | 6 + > pym/portage/util/futures/retry.py| 178 > +++ > 3 files changed, 331 insertions(+) > create mode 100644 pym/portage/tests/util/futures/test_retry.py > create mode 100644 pym/portage/util/futures/retry.py > > diff --git a/pym/portage/tests/util/futures/test_retry.py > b/pym/portage/tests/util/futures/test_retry.py > new file mode 100644 > index 0..7641e4e92 > --- /dev/null > +++ b/pym/portage/tests/util/futures/test_retry.py > @@ -0,0 +1,147 @@ > +# Copyright 2018 Gentoo Foundation > +# Distributed under the terms of the GNU General Public License v2 > + > +import functools > + > +try: > + import threading > +except ImportError: > + import dummy_threading as threading > + > +from portage.tests import TestCase > +from portage.util._eventloop.global_event_loop import global_event_loop > +from portage.util.backoff import RandomExponentialBackoff > +from portage.util.futures.futures import TimeoutError > +from portage.util.futures.retry import retry > +from portage.util.futures.wait import wait > +from portage.util.monotonic import monotonic > + > + > +class SucceedLaterException(Exception): > + pass > + > + > +class SucceedLater(object): > + """ > + A callable object that succeeds some duration of time has passed. > + """ > + def __init__(self, duration): > + self._succeed_time = monotonic() + duration > + > + def __call__(self): > + remaining = self._succeed_time - monotonic() > + if remaining > 0: > + raise SucceedLaterException('time until success: > {} seconds'.format(remaining)) > + return 'success' > + > + > +class SucceedNeverException(Exception): > + pass > + > + > +class SucceedNever(object): > + """ > + A callable object that never succeeds. > + """ > + def __call__(self): > + raise SucceedNeverException('expected failure') > + > + > +class HangForever(object): > + """ > + A callable object that sleeps forever. > + """ > + def __call__(self): > + threading.Event().wait() > + > + > +class RetryTestCase(TestCase): > + def testSucceedLater(self): > + loop = global_event_loop() > + func = SucceedLater(1) > + func_coroutine = functools.partial(loop.run_in_executor, > None, func) > + decorator = retry(try_max=, > + delay_func=RandomExponentialBackoff(multiplier=0.1, > base=2)) > + decorated_func = decorator(func_coroutine) > + result = loop.run_until_complete(decorated_func()) > + self.assertEqual(result, 'success') > + > + def testSucceedNever(self): > + loop = global_event_loop() > + func = SucceedNever() > + func_coroutine = functools.partial(loop.run_in_executor, > None, func) > + decorator = retry(try_max=4, try_timeout=None, > + delay_func=RandomExponentialBackoff(multiplier=0.1, > base=2)) > + decorated_func = decorator(func_coroutine) > + done, pending = loop.run_until_complete(wait([ > decorated_func()])) > + self.assertEqual(len(done), 1) > + self.assertTrue(isinstance(done[0].exception().__cause__, > SucceedNeverException)) > + > + def testSucceedNeverReraise(self): > + loop = global_event_loop() > + func = SucceedNever() > + func_coroutine = functools.partial(loop.run_in_executor, > None, func) > + decorator = retry(reraise=True, try_max=4, > try_timeout=None, > + delay_func=RandomExponentialBackoff(multiplier=0.1, > base=2)) > + decorated_func = decorator(func_coroutine) > + done, pending = loop.run_until_complete(wait([ > decorated_func()])) > + self.assertEqual(len(done), 1) > + self.assertTrue(isinstance(done[0].exception(), > SucceedNeverException)) > + > + def testHangForever(self): > + loop = global_event_loop() > + func = HangForever() > +
Re: [gentoo-portage-dev] [PATCH 1/4] Add ForkExecutor (bug 649588)
lgtm. On Sat, Mar 31, 2018 at 10:46 PM, Zac Medicowrote: > This is useful for asynchronous operations that we might > need to cancel if they take too long, since (concurrent. > futures.ProcessPoolExecutor tasks are not cancellable). > This ability to cancel tasks makes this executor useful > as an alternative to portage.exception.AlarmSignal. > > Also add an asyncio-compatible EventLoop.run_in_executor > method the uses ForkExecutor as the default executor, > which will later be used to implement the corresponding > asyncio.AbstractEventLoop run_in_executor method. > > Bug: https://bugs.gentoo.org/649588 > --- > pym/portage/util/_eventloop/EventLoop.py | 45 - > pym/portage/util/futures/executor/__init__.py | 0 > pym/portage/util/futures/executor/fork.py | 130 > ++ > 3 files changed, 174 insertions(+), 1 deletion(-) > create mode 100644 pym/portage/util/futures/executor/__init__.py > create mode 100644 pym/portage/util/futures/executor/fork.py > > diff --git a/pym/portage/util/_eventloop/EventLoop.py > b/pym/portage/util/_eventloop/EventLoop.py > index f472a3dae..1574a6837 100644 > --- a/pym/portage/util/_eventloop/EventLoop.py > +++ b/pym/portage/util/_eventloop/EventLoop.py > @@ -24,6 +24,7 @@ except ImportError: > import portage > portage.proxy.lazyimport.lazyimport(globals(), > 'portage.util.futures.futures:_EventLoopFuture', > + 'portage.util.futures.executor.fork:ForkExecutor', > ) > > from portage import OrderedDict > @@ -122,6 +123,7 @@ class EventLoop(object): > self._idle_callbacks = OrderedDict() > self._timeout_handlers = {} > self._timeout_interval = None > + self._default_executor = None > > self._poll_obj = None > try: > @@ -721,6 +723,46 @@ class EventLoop(object): > return self._handle(self.timeout_add( > delay * 1000, self._call_soon_callback(callback, > args)), self) > > + def run_in_executor(self, executor, func, *args): > + """ > + Arrange for a func to be called in the specified executor. > + > + The executor argument should be an Executor instance. The > default > + executor is used if executor is None. > + > + Use functools.partial to pass keywords to the *func*. > + > + @param executor: executor > + @type executor: concurrent.futures.Executor or None > + @param func: a function to call > + @type func: callable > + @return: a Future > + @rtype: asyncio.Future (or compatible) > + """ > + if executor is None: > + executor = self._default_executor > + if executor is None: > + executor = ForkExecutor(loop=self) > + self._default_executor = executor > + return executor.submit(func, *args) > + > + def close(self): > + """Close the event loop. > + > + This clears the queues and shuts down the executor, > + and waits for it to finish. > + """ > + executor = self._default_executor > + if executor is not None: > + self._default_executor = None > + executor.shutdown(wait=True) > + > + if self._poll_obj is not None: > + close = getattr(self._poll_obj, 'close') > + if close is not None: > + close() > + self._poll_obj = None > + > > _can_poll_device = None > > @@ -782,10 +824,11 @@ class _epoll_adapter(object): > that is associated with an epoll instance will close automatically > when > it is garbage collected, so it's not necessary to close it > explicitly. > """ > - __slots__ = ('_epoll_obj',) > + __slots__ = ('_epoll_obj', 'close') > > def __init__(self, epoll_obj): > self._epoll_obj = epoll_obj > + self.close = epoll_obj.close > > def register(self, fd, *args): > self._epoll_obj.register(fd, *args) > diff --git a/pym/portage/util/futures/executor/__init__.py > b/pym/portage/util/futures/executor/__init__.py > new file mode 100644 > index 0..e69de29bb > diff --git a/pym/portage/util/futures/executor/fork.py > b/pym/portage/util/futures/executor/fork.py > new file mode 100644 > index 0..9cd1db2ca > --- /dev/null > +++ b/pym/portage/util/futures/executor/fork.py > @@ -0,0 +1,130 @@ > +# Copyright 2018 Gentoo Foundation > +# Distributed under the terms of the GNU General Public License v2 > + > +import collections > +import functools > +import multiprocessing > +import os > +import sys > +import traceback > + > +from
Re: [gentoo-portage-dev] [PATCH] INSTALL_MASK: honor install time config for binary packages (bug 651952)
W dniu czw, 29.03.2018 o godzinie 15∶34 -0700, użytkownik Zac Medico napisał: > For binary packages, honor the INSTALL_MASK configuration that > exists at install time, since it might differ from the build time > setting. > > Fixes: 3416876c0ee7 ("{,PKG_}INSTALL_MASK: python implementation") > Bug: https://bugs.gentoo.org/651952 > --- > bin/misc-functions.sh| 23 +++ > bin/phase-functions.sh | 10 +- > pym/portage/dbapi/vartree.py | 5 + > 3 files changed, 29 insertions(+), 9 deletions(-) > > diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh > index 26f589915..a6330ee93 100755 > --- a/bin/misc-functions.sh > +++ b/bin/misc-functions.sh > @@ -323,6 +323,29 @@ postinst_qa_check() { > done < <(printf "%s\0" "${qa_checks[@]}" | LC_ALL=C sort -u -z) > } > > +preinst_mask() { > + # Remove man pages, info pages, docs if requested. This is > + # implemented in bash in order to respect INSTALL_MASK settings > + # from bashrc. > + local f x > + for f in man info doc; do > + if has no${f} ${FEATURES}; then > + INSTALL_MASK+=" /usr/share/${f}" > + fi > + done > + > + # Store modified variables in build-info. > + cd "${PORTAGE_BUILDDIR}"/build-info || die > + set -f > + > + IFS=$' \t\n\r' > + for f in INSTALL_MASK; do This loop along with the whole indirection is entirely pointless, given that you're processing exactly one variable. > + x=$(echo -n ${!f}) > + [[ -n ${x} ]] && echo "${x}" > "${f}" There's probably no point in this [[ -n ... ]], as that: a. requires you to special-handle missing INSTALL_MASK file, while it's easier to just ensure that it's there (and I think you requested the same thing from me before you rewritten my commit into breakage), b. makes it impossible to distinguish packages from before INSTALL_MASK storing was added from those where it is empty. > + done > + set +f > +} > + > preinst_sfperms() { > if [ -z "${D}" ]; then >eerror "${FUNCNAME}: D is unset" > diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh > index bdae68f79..3de8d01b5 100644 > --- a/bin/phase-functions.sh > +++ b/bin/phase-functions.sh > @@ -661,14 +661,6 @@ __dyn_install() { > set -f > local f x > > - # remove man pages, info pages, docs if requested > - for f in man info doc; do > - if has no${f} ${FEATURES} && \ > - ! has "/usr/share/${f}" ${INSTALL_MASK}; then > - INSTALL_MASK+=" /usr/share/${f}" > - fi > - done > - > IFS=$' \t\n\r' > for f in CATEGORY DEFINED_PHASES FEATURES INHERITED IUSE \ > PF PKGUSE SLOT KEYWORDS HOMEPAGE DESCRIPTION \ > @@ -676,7 +668,7 @@ __dyn_install() { > CXXFLAGS EXTRA_ECONF EXTRA_EINSTALL EXTRA_MAKE \ > LDFLAGS LIBCFLAGS LIBCXXFLAGS QA_CONFIGURE_OPTIONS \ > QA_DESKTOP_FILE QA_PREBUILT PROVIDES_EXCLUDE REQUIRES_EXCLUDE \ > - INSTALL_MASK PKG_INSTALL_MASK; do > + PKG_INSTALL_MASK; do > > x=$(echo -n ${!f}) > [[ -n $x ]] && echo "$x" > $f > diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py > index 378d42dc0..a136c38f1 100644 > --- a/pym/portage/dbapi/vartree.py > +++ b/pym/portage/dbapi/vartree.py > @@ -3846,6 +3846,11 @@ class dblink(object): > # be useful to avoid collisions in some scenarios. > # We cannot detect if this is needed or not here as > INSTALL_MASK can be > # modified by bashrc files. > + phase = MiscFunctionsProcess(background=False, > + commands=["preinst_mask"], phase="preinst", > + scheduler=self._scheduler, settings=self.settings) > + phase.start() > + phase.wait() > try: > with io.open(_unicode_encode(os.path.join(inforoot, > "INSTALL_MASK"), > encoding=_encodings['fs'], errors='strict'), -- Best regards, Michał Górny
Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)
W dniu sob, 31.03.2018 o godzinie 19∶46 -0700, użytkownik Zac Medico napisał: > Since key refresh is prone to failure, retry using exponential > backoff with random jitter. This adds the following sync-openpgp-* > configuration settings: > > sync-openpgp-key-refresh-retry-count = 40 > > Maximum number of times to retry key refresh if it fails. Between > each key refresh attempt, there is an exponential delay with a > constant multiplier and a uniform random multiplier between 0 and 1. > > sync-openpgp-key-refresh-retry-delay-exp-base = 2 > > The base of the exponential expression. The exponent is the number > of previous refresh attempts. > > sync-openpgp-key-refresh-retry-delay-max = 60 > > Maximum delay between each retry attempt, in units of seconds. This > places a limit on the length of the exponential delay. > > sync-openpgp-key-refresh-retry-delay-mult = 4 > > Multiplier for the exponential delay. > > sync-openpgp-key-refresh-retry-overall-timeout = 1200 > > Combined time limit for all refresh attempts, in units of seconds. > > Bug: https://bugs.gentoo.org/649276 > > Zac Medico (4): > Add ForkExecutor (bug 649588) > Add ExponentialBackoff and RandomExponentialBackoff > Add retry decorator (API inspired by tenacity) > rsync: add key refresh retry (bug 649276) > > cnf/repos.conf| 5 + > man/portage.5 | 19 +++ > pym/portage/repository/config.py | 22 > pym/portage/sync/modules/rsync/rsync.py | 16 ++- > pym/portage/sync/syncbase.py | 85 +++- > pym/portage/tests/util/futures/test_retry.py | 147 + > pym/portage/util/_eventloop/EventLoop.py | 45 ++- > pym/portage/util/backoff.py | 48 +++ > pym/portage/util/futures/executor/__init__.py | 0 > pym/portage/util/futures/executor/fork.py | 130 +++ > pym/portage/util/futures/futures.py | 6 + > pym/portage/util/futures/retry.py | 178 > ++ > 12 files changed, 697 insertions(+), 4 deletions(-) > create mode 100644 pym/portage/tests/util/futures/test_retry.py > create mode 100644 pym/portage/util/backoff.py > create mode 100644 pym/portage/util/futures/executor/__init__.py > create mode 100644 pym/portage/util/futures/executor/fork.py > create mode 100644 pym/portage/util/futures/retry.py > This essentially looks like ~700 lines of code to try to workaround broken networking. I would rather try to do that using 5 lines of code but that's just me, and my programs aren't enterprise quality. I just hope it actually solves as many problems as it's going to introduce. -- Best regards, Michał Górny
Re: [gentoo-dev] rfc: empty directories in ${D}
On Sat, 31 Mar 2018 20:46:56 +0100 Andrey Utkinwrote: > Right, I am not aware why PMS has left this explicitly undefined. Because when we wrote the spec, Portage's actual behaviour on empty directories was extremely tricky to define and depended upon things like whether you were upgrading or reinstalling a package. -- Ciaran McCreesh