Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, Jan 15, 2014 at 5:49 AM, William Hubbs willi...@gentoo.org wrote: Also, there is a substantial number of packages which contain only python code (or perl, ruby), or only LaTeX classes, or only documentation. It makes no sense to test them on each arch separately. I think maintainers should be allowed to stabilize such packages (with no compiled code) on all arches. There is a reason we don't do this, back in Gentoo history somewhere, but I don't remember what it was. If someone can tell us why this isn't allowed I am all ears. Otherwise, I could agree on this point as well. Yeah, as the python team lead, I feel we could definitely stick some policy bits in (almost) all python packages that says stable for one arch means stable for all arches. Sure, there'd be some fallout, but I suspect that would be very limited, and in return only one arch tester (or the maintainer!) can stabilize for all architectures. Cheers, Dirkjan
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Tue, 2014-01-14 at 22:49 -0600, William Hubbs wrote: Also, there is a substantial number of packages which contain only python code (or perl, ruby), or only LaTeX classes, or only documentation. It makes no sense to test them on each arch separately. I think maintainers should be allowed to stabilize such packages (with no compiled code) on all arches. There is a reason we don't do this, back in Gentoo history somewhere, but I don't remember what it was. If someone can tell us why this isn't allowed I am all ears. Otherwise, I could agree on this point as well. Speaking for ruby I have seen various arch-related bugs in pure ruby code. It doesn't happen a lot (maybe 1% of stable requests) but it is also not predictable. I also like the second set of eyes verifying what we've done as part of marking a package stable, so I probably would still file bugs rather than marking stuff stable myself. Hans
Re: [gentoo-dev] rfc: revisiting our stabilization policy
Dnia 2014-01-14, o godz. 15:37:19 William Hubbs willi...@gentoo.org napisał(a): I want comments wrt two ideas: 1. I think maintainers should be able to stabilize their packages on arch's they have access to. I think this is allowed by some arch teams, but I think it would be good to formalize it. I think we'd use more feedback from the 'other' arch teams before agreeing on this. Some arches may have a pretty tricky issues that could affect stabilization but which average developer may be not aware of. Maybe it'd be good if each arch team had a wiki page explaining the testing process for their arch. We should also make it clear that the developer is supposed to test the package on a pure stable system to avoid misunderstandings. 2. I would like to see the policy below applied to all arch's [2]. [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt Honestly, this sounds like a very bad idea to me. Even on the minor architectures. TLDR: users will end up running unsupported mixed-arch systems and stabilizing the package again sometime wouldn't make much sense. Dropping the stable keyword on a package means that user either: 1) has to remove the package and either find an alternative or lose particular features completely. And unlike with regular package.mask, he won't get any tips from us. In fact, this policy makes it possible to kill, say, the last graphical word processor on the arch. 2) has to add package.accept_keywords entry for the package. Which means turning a pure stable system into an unsupported mixed-keyword system. Considering portage behavior, I think that 2) is much more likely. Now, the keyword may be added per-version or per-package. If it's added per-version, user simply ends up sticking to another single version until he thinks of upgrading the package manually. If it's added per-package, the keyword usually persists on the user's system. When we bring the stable keywords to the package again, user would have to notice that and remove his override. How likely is that going to happen? So, in the end once we remove stable keyword from a package, most users add ~arch keyword and future stable keyword on the package becomes meaningless. I'd rather go for removing stable keywords from all packages. This would at least make turning the architecture back stable easy for users. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 01:37, William Hubbs пишет: I want comments wrt two ideas: 1. I think maintainers should be able to stabilize their packages on arch's they have access to. I think this is allowed by some arch teams, but I think it would be good to formalize it. 2. I would like to see the policy below applied to all arch's [2]. Thoughts? William [1] http://bugs.gentoo.org/487332 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt amd64 and x86 arch teams have agreement, that maintainers can stabilize their packages if they know how to properly test them. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 03:11, William Hubbs пишет: The status quo is not good, because we are forced to keep old, and potentially buggy, versions of software around longer than necessary. arch team member opinion But both of suggested solutions will break the whole idea of stabling. Dropping packages to unstable on regular basis will annoy our users. If we have old stable package, it builds and works correctly in new environment(new gcc, glibc etc), then i do not see the point in rushing things up, unless there are some more critical things, that needs new version of this package. Stable should be reasonable. Each new version of package contains bugfixes, it is true. But we should note, that new versions(even so-called bugfix releases) can also bring new bugs. /arch team member -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 01:37, William Hubbs пишет: All, It is becoming more and more obvious that we do not have enough manpower on the arch teams, even some of the ones we consider major arch's, to keep up with stabilization requests. For example, there is this bug [1], which is blocking the stabilization of several important packages. And by the way, the only arches left there are ppc and ppc64, which are NOT major ones. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 06:42, Tom Wijsman пишет: And for that occasional mis-guess, *boohoo*, the user can just file a bug; which ironically even happens occasionally for stable packages. If we blindly approves increasing of such mis-guesses, then our QA level in arch teams will down below the apropriate level IMO. And this is not good first of all for our stable users. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, Jan 15, 2014 at 4:54 AM, Michał Górny mgo...@gentoo.org wrote: 2) has to add package.accept_keywords entry for the package. Which means turning a pure stable system into an unsupported mixed-keyword system. As opposed to an unsupported pure stable system or an unsupported pure unstable system? I doubt anybody runs a pure stable system, and if they do I doubt their experience is much different from those who run mixed keywords. Of course, having random system packages drop stable keywords from time to time isn't going to help anything in that department. Obviously maintainers should keep stable system packages around as long as they can. However, there needs to be some way to deal with cases where their stabilization gets held up on a major arch. If we don't have the manpower to stabilize newer versions, what makes anybody think that maintainers have the manpower to support ancient versions? The have our cake and eat it too solution is to obtain more manpower. That should be done without question, but for whatever reason it never actually happens, so we need to think about what the alternative is. If talking about it scares more people into volunteering so that it isn't necessary all the better... Given constrained manpower the options basically are some variation on: 1. Status quo - for some packages stable gets REALLY old, and likely has problems that maintainers ignore. You can't force somebody to maintain something any more than you can force somebody to test something. 2. We become more liberal with stabilizing newer versions. Lots of variations on this, but the bottom line is that packages get stabilized that would otherwise not be. 3. We drop stable keywords on packages. Lots of variations on this as well, but the result is mixed-arch systems or dropping stable for the whole arch. I don't think #1 is really the answer - it just results in less-visible problems and a stable that is probably works worse than either #2 or #3 would yield. #2 has the advantage that it gives us more control over what gets installed on stable systems and you don't end up with mixed keywords. The downside to #2 is that the user doesn't have as much visibility into which packages are really stable. Maybe an in-between quality tier would help (if you're going to run a mixed keyword system, at least use this version). #3 has the advantage that it makes the problem directly visible to the user. However, the solution ends up being entirely user-discretion. Some users end up on ~arch for life, others do it version-by-version in which case they end up on various versions. Personally I run stable and when I need to run unstable on a package I tend to use cat/foo-major+1 syntax so that I'm tracking unstable until the next major version and then I get a chance to revisit the decision - usually stable catches up by then. The only nice solution is more manpower. I'd like a pony to go with that as well... Rich
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, Jan 15, 2014 at 03:30:39PM +0400, Sergey Popov wrote: 15.01.2014 01:37, William Hubbs пишет: All, It is becoming more and more obvious that we do not have enough manpower on the arch teams, even some of the ones we consider major arch's, to keep up with stabilization requests. For example, there is this bug [1], which is blocking the stabilization of several important packages. And by the way, the only arches left there are ppc and ppc64, which are NOT major ones. Sparc is also still on that bug, and according to the council decision I sited, these arch's are still treated like major arch's. Wrt your comment about x86 and amd64 having agreements that maintainers can stabilize packages on those arch's, I thought amd64 did, but I didn't know about x86. Formal policy says that all stabilizations must be done by arch teams unless you have special arrangements with them [1], so my questions still stand. 1. Should we make it policy that maintainers can stabilize packages on arch's they have access to? 2. See Rich's message in this thread for my other concern; he spells it out pretty well -- what should we do about architectures the maintainer does not have access to? 3. Also, another interesting question has come up in this thread, that of non-binary packages. Should we give maintainers the option of stabilizing them on all arch's themselves? William [1] http://devmanual.gentoo.org/keywording/index.html signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, 15 Jan 2014 15:33:28 +0400 Sergey Popov pinkb...@gentoo.org wrote: 15.01.2014 06:42, Tom Wijsman пишет: And for that occasional mis-guess, *boohoo*, the user can just file a bug; which ironically even happens occasionally for stable packages. If we blindly approves increasing of such mis-guesses, then our QA level in arch teams will down below the apropriate level IMO. And this is not good first of all for our stable users. What I'm saying is that even on arch team stabilized ebuilds bugs getting filed happens as well; so, it happening for a maintainer stabilized ebuild wouldn't be so problematic from that perspective. But, indeed, it depends on which arch team procedure efforts the maintainer actually applies; on an own arch it is quite possible for the maintainer to be near the QA level of the arch team, whereas not having access to a certain architecture can indeed become problematic. So, for the arches the maintainers do have, it depends on what the maintainers do as much as what the arch teams do; and to be realistic arch teams might not always do everything as intended, especially under time pressure. In my opinion, a maintainer would even spend more time. As for arches the maintainer does not have, the available machines might be usable; though the doubt is whether they have enough power. Most of this discussion is hypothetical assuming stabilization stays (or continues to grow to be more) problematic; who knows we might get to see the opposite effect that this thread yields some new arch team members, which could make what we've discussed here not necessary in the short term run. It is clear everyone wants to hold on to the arch teams. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, 15 Jan 2014 15:40:20 +0400 Sergey Popov pinkb...@gentoo.org wrote: As i said earlier for similar proposals - the one option that i see here to make all things going better - to recruit more people in arch teams/arch testers. Other options lead us to nowhere, when stable will be eliminated or transformed into fake. If eventually our existing approach yields no or worsening results, it would leads us nowhere as well; we can pick that option a few times, but if it doesn't improve anything we'll need to start reconsidering. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On 01/15/2014 10:57 AM, Tom Wijsman wrote: On Wed, 15 Jan 2014 15:33:28 +0400 Sergey Popov pinkb...@gentoo.org wrote: 15.01.2014 06:42, Tom Wijsman пишет: And for that occasional mis-guess, *boohoo*, the user can just file a bug; which ironically even happens occasionally for stable packages. If we blindly approves increasing of such mis-guesses, then our QA level in arch teams will down below the apropriate level IMO. And this is not good first of all for our stable users. What I'm saying is that even on arch team stabilized ebuilds bugs getting filed happens as well; so, it happening for a maintainer stabilized ebuild wouldn't be so problematic from that perspective. But, indeed, it depends on which arch team procedure efforts the maintainer actually applies; on an own arch it is quite possible for the maintainer to be near the QA level of the arch team, whereas not having access to a certain architecture can indeed become problematic. So, for the arches the maintainers do have, it depends on what the maintainers do as much as what the arch teams do; and to be realistic arch teams might not always do everything as intended, especially under time pressure. In my opinion, a maintainer would even spend more time. As for arches the maintainer does not have, the available machines might be usable; though the doubt is whether they have enough power. Most of this discussion is hypothetical assuming stabilization stays (or continues to grow to be more) problematic; who knows we might get to see the opposite effect that this thread yields some new arch team members, which could make what we've discussed here not necessary in the short term run. It is clear everyone wants to hold on to the arch teams. I would like to see the ability for devs to become arch testers for the packages they own on the architectures they have access to. The hard part is enforcing the arch testing guidelines, so maybe this can be considered 'arch tester jr.'. -- -- Matthew Thode (prometheanfire) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
William Hubbs schrieb: Thoughts? William [1] http://bugs.gentoo.org/487332 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt I see 2 cases here: 1. specific or all arch teams allow maintainers to stabilize packages on their own, when they follow the arch team stabilization rules (e.g. having a system running with stable keywords for testing the package). This should not reduce the quality of the stable tree (or only to the small amount, that some arch testers do additional checks the maintainer does not do). Reading through this thread, it seems like amd64 and x86 arch teams already use this policy. This sounds like a reasonable agreement, so i am supporting this too. 2. for arches with no such agreement or where the maintainer does not have the needed hardware to test, no action for a certain amount of time usually means, that the arch team is overloaded with work so the only short- to mid-term solution is to reduce the amount of work resulting in smaller amount of stable packages. So i am voting for maintainers dropping stable keywords after a certain amount of time with no actions (maybe with some notice beforehand). This might result in a mixed arch user setup by default, but imho it is still better to have a smaller stable set of core packages and testing packages on top then having either everything on testing or broken/untested/unsupported packages, which are still claimed to be the opposite with the stable keyword. short summary: -in agreement with arch teams, following stabilization policy and having the needed hardware, maintainers should be able to add stable keywords themselves -if either agreement of arch team or needed hardware is missing, keywords should be dropped, so that after some time the workload matches the abilities of the arch team again. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, Jan 15, 2014 at 07:33:45PM +0100, Thomas Sachau wrote: William Hubbs schrieb: Thoughts? William [1] http://bugs.gentoo.org/487332 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt I see 2 cases here: 1. specific or all arch teams allow maintainers to stabilize packages on their own, when they follow the arch team stabilization rules (e.g. having a system running with stable keywords for testing the package). This should not reduce the quality of the stable tree (or only to the small amount, that some arch testers do additional checks the maintainer does not do). Reading through this thread, it seems like amd64 and x86 arch teams already use this policy. This sounds like a reasonable agreement, so i am supporting this too. 2. for arches with no such agreement or where the maintainer does not have the needed hardware to test, no action for a certain amount of time usually means, that the arch team is overloaded with work so the only short- to mid-term solution is to reduce the amount of work resulting in smaller amount of stable packages. So i am voting for maintainers dropping stable keywords after a certain amount of time with no actions (maybe with some notice beforehand). This might result in a mixed arch user setup by default, but imho it is still better to have a smaller stable set of core packages and testing packages on top then having either everything on testing or broken/untested/unsupported packages, which are still claimed to be the opposite with the stable keyword. short summary: -in agreement with arch teams, following stabilization policy and having the needed hardware, maintainers should be able to add stable keywords themselves -if either agreement of arch team or needed hardware is missing, keywords should be dropped, so that after some time the workload matches the abilities of the arch team again. When you say drop keywords do you mean: 1) revert the old version back to ~arch or 2) remove the old version. As a maintainer, I would rather do 2, because I do not want to backport fixes to the old version. William signature.asc Description: Digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Tuesday 14 January 2014 22:37:19 William Hubbs wrote: I think we need a global policy that either helps keep the stable tree up to date or reverts an architecture to ~ over time if the arch team can't keep up. As a compromise solution for minor archs, it would be nice if there were a portage feature allowing me to ACCEPT_KEYWORDS those packages that are keyworded both ~arch, and stable on some major arch. For example, on m68k, it would select packages that are keyworded ~m68k amd64, or ~m68k x86, etc. This would allow me to run m68k kinda stable. [Speculation as to applying a similar system more broadly witheld.] -- Ruud
[gentoo-dev] Re: rfc: revisiting our stabilization policy
Rich Freeman posted on Wed, 15 Jan 2014 07:51:49 -0500 as excerpted: Given constrained manpower the options basically are some variation on: 1. Status quo - for some packages stable gets REALLY old, and likely has problems that maintainers ignore. You can't force somebody to maintain something any more than you can force somebody to test something. 2. We become more liberal with stabilizing newer versions. Lots of variations on this, but the bottom line is that packages get stabilized that would otherwise not be. 3. We drop stable keywords on packages. Lots of variations on this as well, but the result is mixed-arch systems or dropping stable for the whole arch. I don't think #1 is really the answer - it just results in less-visible problems and a stable that is probably works worse than either #2 or #3 would yield. #2 has the advantage that it gives us more control over what gets installed on stable systems and you don't end up with mixed keywords. The downside to #2 is that the user doesn't have as much visibility into which packages are really stable. Maybe an in-between quality tier would help (if you're going to run a mixed keyword system, at least use this version). What about a third target-stable keyword? Either arch-teams or package- maintainers (with maintainers getting priority in case of differing viewpoints, just as arch-teams already have priority over full-stable) could set this, and it would let users who wanted to be almost stable but hasn't gotten thru all the hoops just yet arch-testers do just that, see and test almost-stable packages. An open stabilization bug would be required for any target-stable package, and only one target-stable per-slot per-arch would be allowed -- if there's already a target-stable in that slot for that arch, it would need to be removed and either that package fully stabilized or the new one substituted in the target-stable slot. * Note however that different archs could however have different target- stable packages within a slot. This would allow a maintainer to set a minimal version target-stable keyword for lagging archs, while setting a preferred version target-stable keyword for current archs. A maintainer facing lagging archs could then set target-stable appropriately, and could re-assign any bugs on the old-stable version to the arch in question, with comment boilerplate to the effect that the arch is lagging and hasn't updated stable, so they get to deal with bugs for their ancient-stable version. A variant on the theme would be to split the current stable keyword in half, arch-stable and maintainer stable. Any package version keyworded for both would be considered fully stable, but the two halves would then be fully independent, no longer interlocked, and for half-stable packages bugs would be assigned to the stable side with the other side CCed. In this variant, there'd be no limit on the number of package versions that could be half-stabled by either half, just as there can now be multiple stable-keyworded versions, and for that matter, multiple testing versions as well. #3 has the advantage that it makes the problem directly visible to the user. However, the solution ends up being entirely user-discretion. Either target-stable keyword variant above would be directly visible to the end user as well, and of course they could choose full-stable or half- stable on either side as they wished. Some users end up on ~arch for life, others do it version-by-version in which case they end up on various versions. Personally I run stable and when I need to run unstable on a package I tend to use cat/foo-major+1 syntax so that I'm tracking unstable until the next major version and then I get a chance to revisit the decision - usually stable catches up by then. Interesting. Nominally I do ~arch as for me stab?le== , but I do more or less the same thing at the overlay-~arch/unkeyworded/masked/live- level. For my kde packages, for example, I run the overlay and normally package.keyword/package.unmask 4.y. as soon as it's available in the kde overlay (I have scripts that do git log --color --stat --graph $branch ORIG_HEAD.. on the overlays, and routinely run them after syncing so I know what's going on). But since upstream kde doesn't split a branch until the rcs and thus those branches and the kde overlay packages based on them don't appear for the betas, I have to switch to - during the kde betas, and downgrade back to 4.y. when upstream branches and the 4.y. ebuilds appear. I suppose one of these days I'll setup kde-frameworks and be doing about the same for it, too... What's nice is that for kde 4.12. for example, when gentoo/kde was still sorting out the masking/dependencies issues in the overlay due to plasma/workspace being locked to 4.11.x, I was able to grab the 4.11. unmask files from the overlay, do a global search and replace
[gentoo-dev] Re: rfc: revisiting our stabilization policy
Michael Orlitzky posted on Tue, 14 Jan 2014 19:50:30 -0500 as excerpted: As I mentioned in a reply to William, right now I can decide when stuff is broken and keyword the newer versions. The proposal is to force me onto the new versions, which is strictly worse from my perspective. Force?? As I discovered when gentoo/kde forced me into taking semantic-desktop up the beep with early kde 4.11, there's rather less force on gentoo than many realize, certainly as long as upstream is still supporting the options, anyway, one of the reasons I run gentoo.[1] =:^) Every once in awhile I drag an ebuild out of /var/db/pkg/ to put in my overlay, because the ~arch I normally run has moved on and my current version is gone, but the new version is broken (or simply hugely changed and I don't have time to reconfigure ATM), while the stab?le version is just that, stale. Of course the kde-sunset overlay is perhaps the ultimate example of that. Yes, ultimately there will eventually be some forcing as in-tree deps change and keeping the old/overlaid version building and running becomes more of an issue over time, but it'll be a gradual process over a number of years, and the gentooer remains free to pick his pain point and do the migration in his own time, which at minimum, makes it a substantially softer force than would be the case on /most/ distributions. --- [1] In the kde/semantic-desktop case, I diffed package versions with and without the flag and figured out which changes were related to it and which not, creating my own ebuild patches, which I dropped in a tree under /etc/portage/patches.ebuild/, similar to the /etc/portage/patches/ tree. I then hacked up a script to apply those ebuild patches and re- manifest, and added that step to my sync-script. This was all possible, and actually surprisingly easy, because (1) upstream kde still supports the configure options and AFAIK intends to thru all of kde4, and (2) gentoo/kde had the options available at one point, so all I had to do was diff the before and after, and reverse the effect, hard-coding the flag off, where gentoo/kde was was effectively hard-coding it on. Fortunately, before 4.11 went stable, gentoo/kde decided to keep the flags after all, and reintroduced them. So I didn't have to carry my own patches for as long as I had feared I might. But regardless, their forcing semantic-desktop on ~arch and overlay users didn't force /me/ to take it, because I'm an empowered gentooer and one way or another I wasn't taking any such forcing! There efforts underway to do a user- controlled kde-sunset overlay thing, possibly calling it kde-lite, too, thus spreading the work around a bit, but fortunately that ultimately wasn't needed. And if it had come to it, I was beginning to look at other desktops too, as I had tried it previously and was done with kde with semantic-desktop, period, but fortunately that migration didn't have to happen either. =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-dev] Re: rfc: revisiting our stabilization policy
Tom Wijsman posted on Wed, 15 Jan 2014 01:28:09 +0100 as excerpted: Another option (and I don't mean to step on any toes or call anyone out here, these are just examples) may be to just deprecate stabilizing certain software. Packages such as the stuff in app-vim/ or app-emacs/ or some games or some scientific software. For the editor plugins, most people do not get them from the package manager I feel and a Vim plugin requires almost as much arch testing work as a new version of grep, for example... Sounds like a good idea, but how do we translate that to the user; always mark them stable, or always mark them unstable? Do we want users to explicitly accept keywords on these packages? As a ~arch/masked/overlay/live user here (who additionally doesn't even use gentoo kernel ebuilds, preferring direct upstream git and my own scripts), I haven't even checked if it actually happened (looks like gentoo-sources-3.10.x is still stable on x86/amd64, so maybe not), but... There was previous discussion of destable-keywording the kernel. How has that gone? I've always thought that having a stable policy exception that the user actually has to deal with for certain packages, particularly core packages such as the kernel, would be confusing at best. Still, given the upstream development pattern, I couldn't think of a reasonable alternative for the kernel, and agreed with the thread that it may have to be, for packages like that and perhaps google-chrome and firefox, where upstream releases are too close to 30-day and updates are very likely to be security-critical on packages that are net-exposed. So it seemed it had to be, for them, and if that has gone well, perhaps expanding that no-stable policy precedent to things like editor plugins could work better than I might have imagined. The other question then becomes, since ~arch packages are normally masked to stable, how are users exposed to them? What about a file somewhere in profiles that lists all these no-stable packages, such that the PM can (perhaps optionally, I could imagine a setting in make.conf...) list all ~arch versions of those packages on an otherwise stable system as if they were stable, tho possibly marked in some way to indicate that this package isn't a stable-keyword candidate? -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] Re: rfc: revisiting our stabilization policy
On Wed, 15 Jan 2014 23:59:49 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: There was previous discussion of destable-keywording the kernel. How has that gone? That was for vanilla-sources only, because that has restricted to only the latest upstream version; as that makes the version change almost weekly, the package can't undergo our stabilization procedure. I've always thought that having a stable policy exception that the user actually has to deal with for certain packages, particularly core packages such as the kernel, would be confusing at best. Yes, if this would ever happen to gentoo-sources; I'd think the handbook would then need to be updated to mention the necessary extra step, but I think it is not bound to happen any time soon. Still, given the upstream development pattern, I couldn't think of a reasonable alternative for the kernel, and agreed with the thread that it may have to be, for packages like that and perhaps google-chrome and firefox, where upstream releases are too close to 30-day and updates are very likely to be security-critical on packages that are net-exposed. What we do now appears to work fine, critical security bugs cause fast track stabilization if needed; I've backported some security fixes in the past for less critical CVEs in the past, but the main problem here for keeping this up is the lack of manpower on the kernel team. So it seemed it had to be, for them, and if that has gone well, perhaps expanding that no-stable policy precedent to things like editor plugins could work better than I might have imagined. I think it needs to put the accept keywords in a more prominent place if we're going to do this at a wider scale; currently it's in one of those sections that people often don't read due to focusing on continuing with there install instead, eg. they move to some DE guide. The other question then becomes, since ~arch packages are normally masked to stable, how are users exposed to them? They aren't unless they accept keywords for them; which can either be done globally using package.accept_keywords, or locally by listing the package atom in /etc/portage/package.accept_keywords What about a file somewhere in profiles that lists all these no-stable packages, such that the PM can (perhaps optionally, I could imagine a setting in make.conf...) list all ~arch versions of those packages on an otherwise stable system as if they were stable, tho possibly marked in some way to indicate that this package isn't a stable-keyword candidate? If we drop stable versions on a wider scale, we could indeed make the ~arch versions more visible where they currently aren't; we don't want to give the impression that we are removing everything. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, 2014-01-15 at 13:07 -0600, William Hubbs wrote: When you say drop keywords do you mean: 1) revert the old version back to ~arch or 2) remove the old version. As a maintainer, I would rather do 2, because I do not want to backport fixes to the old version. William I'm not sure what he meant by drop keywords either, however, something that I would like to see (with my ARM hat on here) - is, if something is taking a while to stable for a certain arch, remove the keywords except for that existing arch. We actually ran into something along this issue with git. Now, arm is an interesting keyword, because for arm, when something needs to be stabled, we have to test armv4, armv5, armv6, armv6 hardfloat, armv7, armv7 hardfloat, armv7 uclibc. In my testing, one known issue was that git on uclibc did (and still doesn't) work properly starting with git 1.8 - so I noted in the bug that this was the case, and to NOT stable it for arm. Unfortunately, someone else on the ARM team disregarded the note and stabled the new git, then the git maintainers dropped the old versions. Now on arm uclibc, git is entirely broken and unusable. And no, adding more people to the arch team doesn't particularly help, as people are buying more and more armv7 so they test that, but not the rest of the versions - and no one wants to buy the older hardware because it's so slow - we know it's slow, that's why it takes time. I'd have definitely preferred that for git, that the 1.7 version stuck around with just KEYWORDS=-* arm (and maybe even stabling 1.8 but leaving 1.7 in masked?) - I realize it was a bit of a special case because of the new git eclass. Unfortunately, debugging what's going on, is a bit above me, and the only other person I know who can/does work on it, is blueness, and he's quite busy.
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Wed, Jan 15, 2014 at 06:58:27PM -0600, Steev Klimaszewski wrote: We actually ran into something along this issue with git. Now, arm is an interesting keyword, because for arm, when something needs to be stabled, we have to test armv4, armv5, armv6, armv6 hardfloat, armv7, armv7 hardfloat, armv7 uclibc. In my testing, one known issue was that git on uclibc did (and still doesn't) work properly starting with git 1.8 - so I noted in the bug that this was the case, and to NOT stable it for arm. Unfortunately, someone else on the ARM team disregarded the note and stabled the new git, then the git maintainers dropped the old versions. Now on arm uclibc, git is entirely broken and unusable. Ugh, this does suck. Wasn't there a proposal years ago to include the libc in the keyword? -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Thu, 2014-01-16 at 02:32 +, Robin H. Johnson wrote: In my testing, one known issue was that git on uclibc did (and still doesn't) work properly starting with git 1.8 - so I noted in the bug that this was the case, and to NOT stable it for arm. Unfortunately, someone else on the ARM team disregarded the note and stabled the new git, then the git maintainers dropped the old versions. Now on arm uclibc, git is entirely broken and unusable. Ugh, this does suck. It does, though it's specific to arm uclibc, as it works fine on amd64/x86 uclibc. And unfortunately, it seems like this type of thing is what people are proposing we move towards. Instead of working but old, not having access to the software at all. I know it's not the norm, nor is it typical, but the chance of this happening does exist, and I can't see how anyone would say, well, that's just the chance that people should take, unless they've never been bitten by a bug like this. Wasn't there a proposal years ago to include the libc in the keyword? There may have been, I'm not sure that's really the right step either though.
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 19:30, William Hubbs пишет: On Wed, Jan 15, 2014 at 03:30:39PM +0400, Sergey Popov wrote: 15.01.2014 01:37, William Hubbs пишет: All, It is becoming more and more obvious that we do not have enough manpower on the arch teams, even some of the ones we consider major arch's, to keep up with stabilization requests. For example, there is this bug [1], which is blocking the stabilization of several important packages. And by the way, the only arches left there are ppc and ppc64, which are NOT major ones. Sparc is also still on that bug, and according to the council decision I sited, these arch's are still treated like major arch's. Well, to be honest, personally i consider only amd64 and x86(and maybe arm) as major arches, other are minor in my eyes. Council decision is more about arches, that crucially lacks manpower. Wrt your comment about x86 and amd64 having agreements that maintainers can stabilize packages on those arch's, I thought amd64 did, but I didn't know about x86. It's not mentioned, yeah, i was not aware about it for some time. Probably it should be mentioned in Gentoo Development Guide. Formal policy says that all stabilizations must be done by arch teams unless you have special arrangements with them [1], so my questions still stand. 1. Should we make it policy that maintainers can stabilize packages on arch's they have access to? 2. See Rich's message in this thread for my other concern; he spells it out pretty well -- what should we do about architectures the maintainer does not have access to? 3. Also, another interesting question has come up in this thread, that of non-binary packages. Should we give maintainers the option of stabilizing them on all arch's themselves? 1. If you know how to test it properly, know arch-specific problems(aligning, endianness, ABI breakage) and how to fix it - then, probably yes. But usually maintainers are bored to do proper testing. 2. I think - no. You can not test it - you can not stabilize it, period. 3. If code is interpreted rather then compiled, it does not matter that it is properly ported on minor arches. I knew dozens of examples with Perl and Python packages(not sure about Ruby, but Hans said that it happens with it too). So, i would not treat such packages differently. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 21:04, Tom Wijsman пишет: On Wed, 15 Jan 2014 15:40:20 +0400 Sergey Popov pinkb...@gentoo.org wrote: As i said earlier for similar proposals - the one option that i see here to make all things going better - to recruit more people in arch teams/arch testers. Other options lead us to nowhere, when stable will be eliminated or transformed into fake. If eventually our existing approach yields no or worsening results, it would leads us nowhere as well; we can pick that option a few times, but if it doesn't improve anything we'll need to start reconsidering. It can not go to no result, unless we have no breakages in stable, stable REMAINS stable. If it contains old, but working software - then it is stable. As i said earlier, problem begins when we NEED to stabilize something to prevent breakages and arch teams are slow. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] rfc: revisiting our stabilization policy
15.01.2014 22:33, Thomas Sachau пишет: William Hubbs schrieb: Thoughts? William [1] http://bugs.gentoo.org/487332 [2] http://www.gentoo.org/proj/en/council/meeting-logs/20130917-summary.txt I see 2 cases here: 1. specific or all arch teams allow maintainers to stabilize packages on their own, when they follow the arch team stabilization rules (e.g. having a system running with stable keywords for testing the package). This should not reduce the quality of the stable tree (or only to the small amount, that some arch testers do additional checks the maintainer does not do). Reading through this thread, it seems like amd64 and x86 arch teams already use this policy. This sounds like a reasonable agreement, so i am supporting this too. 2. for arches with no such agreement or where the maintainer does not have the needed hardware to test, no action for a certain amount of time usually means, that the arch team is overloaded with work so the only short- to mid-term solution is to reduce the amount of work resulting in smaller amount of stable packages. So i am voting for maintainers dropping stable keywords after a certain amount of time with no actions (maybe with some notice beforehand). This might result in a mixed arch user setup by default, but imho it is still better to have a smaller stable set of core packages and testing packages on top then having either everything on testing or broken/untested/unsupported packages, which are still claimed to be the opposite with the stable keyword. short summary: -in agreement with arch teams, following stabilization policy and having the needed hardware, maintainers should be able to add stable keywords themselves -if either agreement of arch team or needed hardware is missing, keywords should be dropped, so that after some time the workload matches the abilities of the arch team again. Thanks, for the proposal. IIRC, there was similar backroom agreement in some minor arches, look at how armin76 stabilized packages earlier - sometimes he drops vast amount of keywords on ia64/sparc/m68k to unstable in stabilization requests. And i think we should continue this practice. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
[gentoo-dev] Re: rfc: revisiting our stabilization policy
On 01/16/2014 05:27 PM, Sergey Popov wrote: Thanks, for the proposal. IIRC, there was similar backroom agreement in some minor arches, look at how armin76 stabilized packages earlier - sometimes he drops vast amount of keywords on ia64/sparc/m68k to unstable in stabilization requests. And i think we should continue this practice. I agree completely. The keywords on many packages are historical and do not necessarily represent the current reality. Is it really a good use of our resources to maintain stable keywords for some small auxiliary package? I think every stable request is a good opportunity for each minor arch team to review their keywords for that package.
Re: [gentoo-dev] rfc: revisiting our stabilization policy
On Tue, 14 Jan 2014 20:46:04 -0600 William Hubbs willi...@gentoo.org wrote: s/month/year/ Do you feel the same way then? I have heard of stabilizations taking this long before. I just had to try to pick something reasonable for the discussion. I suppose a compromise would be, instead of removing the old versions, move them back to ~arch for a month then remove them, but that still implies action on your part. William If I need or want a feature or bugfix which isn’t in the newer version, I always have the choice to use ~. If I don’t, why do I care if the package is a year old? I lose none of my time to use the old version, since it does all I want; I lose a nonzero amount of time if I get a version which breaks things (even if the only time I lose is the time it takes to downgrade), so it’s in my best interest to use the stable versions of such packages, even if they’re a month, a year, or three years old. -- Christopher Head signature.asc Description: PGP signature
[gentoo-portage-dev] [PATCH v2] Add repoman check to warn if src_prepare/src_configure are used in EAPI 0/1 and if pkg_pretend is used in EAPI 4. Fixes bug 379491.
--- pym/repoman/checks.py | 29 + 1 file changed, 29 insertions(+) diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py index 85aa065..5c55b0d 100644 --- a/pym/repoman/checks.py +++ b/pym/repoman/checks.py @@ -731,6 +731,21 @@ class DeprecatedHasq(LineCheck): re = re.compile(r'(^|.*\b)hasq\b') error = errors.HASQ_ERROR +# EAPI 2 checks +class Eapi01UndefinedPhases(LineCheck): + repoman_check_name = 'EAPI.incompatible' + src_configprepare_re = re.compile(r'\s*src_(configure|prepare)\s*\(\)') + + def check_eapi(self, eapi): + return eapi in ('0', '1') + + def check(self, num, line): + m = self.src_configprepare__re.match(line) + if m is not None: + return (%s % m.group(1)) + \ +phase is not defined in EAPI=0/1 on line: %d + + # EAPI-3 checks class Eapi3DeprecatedFuncs(LineCheck): repoman_check_name = 'EAPI.deprecated' @@ -745,6 +760,20 @@ class Eapi3DeprecatedFuncs(LineCheck): return ('%s' % m.group(1)) + \ has been deprecated in EAPI=3 on line: %d +# EAPI 4 checks +class Eapi0123UndefinedPhases(LineCheck): + repoman_check_name = 'EAPI.incompatible' + pkg_pretend_re = re.compile(r'\s*pkg_pretend\s*\(\)') + + def check_eapi(self, eapi): + return eapi in ('0', '1', '2', '3') + + def check(self, num, line): + m = self.pkg_pretend_re.match(line) + if m is not None: + return (%s % m.group(1)) + \ +phase is not defined in EAPI 4 on line: %d + # EAPI-4 checks class Eapi4IncompatibleFuncs(LineCheck): repoman_check_name = 'EAPI.incompatible' -- 1.8.5.1
Re: [gentoo-portage-dev] [PATCH] Document bugzilla workflow
Am 15.01.2014 17:20, schrieb Tom Wijsman: On Wed, 15 Jan 2014 07:29:19 +0100 Sebastian Luther sebastianlut...@gmx.de wrote: Am 15.01.2014 04:11, schrieb Tom Wijsman: I send the first mail with this wording 8 days ago. Enough time to comment on it. I'd prefer to discuss it on the list. Yes, but not all comments were discussed yet, therefore (dis)agreement on them is missing; and this last thing rather became a topic of discussion due to the work clashes that we saw happen twice. I'd say the clashes occurred because nobody mentioned at all what they are working on. Since people started using IN_PROGRESS to mean I'm working on it, this shouldn't happen again. Yes, I see some commit messages not refer to bugs which is something we will want to avoid; think this might need to go into the commit policy. There's nothing wrong with fixing/implementing something that nobody filed a bug about. The way it was is to not care about them at all. There was no agreement on the the other thread if these things should be used or not. So I left it vague so everyone could use it, but they are not forced to. Hmm, could this result in conflicting usage of these? Maybe, but I'd first see if the usage patterns converge to something that makes everyone happy. +There are a number of bugs named [TRACKER] * that collect bugs +for specific topics. Confirmed bugs should be marked as blocking +these tracker bugs if appropriate. For clarity, it should be mentioned that this does not mean to block the tracker for the next version; this could be misinterpreted. Considering that the tracker gets renamed, I'm not sure what you mean here. As you are confused yourself by misinterpreting what you have written, you demonstrate the case for the need of clarity here; this is not about the next version tracker or it being renamed at all, it's about all other trackers that are not version trackers. The part of the policy quoted here doesn't make that clear, it had me puzzling for a moment too when I first read that; I think you were puzzled too now... Sorry, I failed to properly read what you quoted. I think once you know that these other trackers exist, it's clear. If you want something added there, that's fine with me too. Sebastian
Re: [gentoo-portage-dev] [PATCH] Document bugzilla workflow
Am 15.01.2014 19:41, schrieb Tom Wijsman: Yes, I see some commit messages not refer to bugs which is something we will want to avoid; think this might need to go into the commit policy. There's nothing wrong with fixing/implementing something that nobody filed a bug about. Sorry, consider the common case where a bug was filed but the commit message does not refer to that bug. Also note that I am trying to refer to the ChangeLog of Portage itself, not that of the ebuild; thus I mean the commit messages for the Portage repo, not to the Portage tree. Not sure if we're talking about the same things. 1) If you fix something that has a bug, you should refer to that in the git commit message. 2) There's nothing wrong with a git commit message that does not refer to a bug, if there is no bug filed. The whole point of documenting it in a workflow is to make it converge; Not the converge I meant. What I meant was to allow people to test different styles and hope that the one that works best will be adopted by everyone at some point. Once that happens you can document that style. if you instead leave things unclear or undocumented, you have no guaranteed convergence and might even see a disconvergence. Yes, maybe. One then needs to see if that is a problem and if it is then force everyone to use one style. It's already making people unhappy right now; because as it is documented now, it is turned from the meaningful experience that the previous Portage team had before to something that is meaningless. It is a regression in checking the list of bugs that block the tracker, as the states of the bugs no longer have a value as it is documented now. Previously the bug state was not used at all. There is no regression.
[gentoo-portage-dev] [PATCH v3] Add repoman check to warn if src_prepare/src_configure are used in EAPI 0/1 and if pkg_pretend is used in EAPI 4. Fixes bug 379491.
--- pym/repoman/checks.py | 29 + 1 file changed, 29 insertions(+) Ignore v2, I apparently didn't commit all of my changes and so that patch won't work. Undid the compression of the src_prepare/src_configure regex, because the way that the regex is set up means that it would output prepare phase is not defined in EAPI... instead of src_prepare and I feel that it's more intuitive to match the full name of the function instead of tacking src_ in front of the output. diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py index 85aa065..c6860d8 100644 --- a/pym/repoman/checks.py +++ b/pym/repoman/checks.py @@ -731,6 +731,21 @@ class DeprecatedHasq(LineCheck): re = re.compile(r'(^|.*\b)hasq\b') error = errors.HASQ_ERROR +# EAPI 2 checks +class Eapi01UndefinedPhases(LineCheck): + repoman_check_name = 'EAPI.incompatible' + src_configprepare_re = re.compile(r'\s*(src_configure|src_prepare)\s*\(\)') + + def check_eapi(self, eapi): + return eapi in ('0', '1') + + def check(self, num, line): + m = self.src_configprepare_re.match(line) + if m is not None: + return ('%s' % m.group(1)) + \ +phase is not defined in EAPI 2 on line: %d + + # EAPI-3 checks class Eapi3DeprecatedFuncs(LineCheck): repoman_check_name = 'EAPI.deprecated' @@ -745,6 +760,20 @@ class Eapi3DeprecatedFuncs(LineCheck): return ('%s' % m.group(1)) + \ has been deprecated in EAPI=3 on line: %d +# EAPI 4 checks +class Eapi0123UndefinedPhases(LineCheck): + repoman_check_name = 'EAPI.incompatible' + pkg_pretend_re = re.compile(r'\s*(pkg_pretend)\s*\(\)') + + def check_eapi(self, eapi): + return eapi in ('0', '1', '2', '3') + + def check(self, num, line): + m = self.pkg_pretend_re.match(line) + if m is not None: + return ('%s' % m.group(1)) + \ +phase is not defined in EAPI 4 on line: %d + # EAPI-4 checks class Eapi4IncompatibleFuncs(LineCheck): repoman_check_name = 'EAPI.incompatible' -- 1.8.5.1
[gentoo-portage-dev] [PATCH v4] Add repoman check to warn if src_prepare/src_configure are used in EAPI 0/1 and if pkg_pretend is used in EAPI 4. Fixes bug 379491.
--- pym/repoman/checks.py | 31 ++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py index 85aa065..c814fa7 100644 --- a/pym/repoman/checks.py +++ b/pym/repoman/checks.py @@ -15,7 +15,7 @@ import repoman.errors as errors import portage from portage.eapi import eapi_supports_prefix, eapi_has_implicit_rdepend, \ eapi_has_src_prepare_and_src_configure, eapi_has_dosed_dohard, \ - eapi_exports_AA + eapi_exports_AA, eapi_has_pkg_pretend class LineCheck(object): Run a check on a line of an ebuild. @@ -731,6 +731,21 @@ class DeprecatedHasq(LineCheck): re = re.compile(r'(^|.*\b)hasq\b') error = errors.HASQ_ERROR +# EAPI 2 checks +class UndefinedSrcPrepareSrcConfigurePhases(LineCheck): + repoman_check_name = 'EAPI.incompatible' + src_configprepare_re = re.compile(r'\s*(src_configure|src_prepare)\s*\(\)') + + def check_eapi(self, eapi): + return not eapi_has_src_prepare_and_src_configure(eapi) + + def check(self, num, line): + m = self.src_configprepare_re.match(line) + if m is not None: + return ('%s' % m.group(1)) + \ +phase is not defined in EAPI 2 on line: %d + + # EAPI-3 checks class Eapi3DeprecatedFuncs(LineCheck): repoman_check_name = 'EAPI.deprecated' @@ -745,6 +760,20 @@ class Eapi3DeprecatedFuncs(LineCheck): return ('%s' % m.group(1)) + \ has been deprecated in EAPI=3 on line: %d +# EAPI 4 checks +class UndefinedPkgPretendPhase(LineCheck): + repoman_check_name = 'EAPI.incompatible' + pkg_pretend_re = re.compile(r'\s*(pkg_pretend)\s*\(\)') + + def check_eapi(self, eapi): + return not eapi_has_pkg_pretend(eapi) + + def check(self, num, line): + m = self.pkg_pretend_re.match(line) + if m is not None: + return ('%s' % m.group(1)) + \ +phase is not defined in EAPI 4 on line: %d + # EAPI-4 checks class Eapi4IncompatibleFuncs(LineCheck): repoman_check_name = 'EAPI.incompatible' -- 1.8.5.1
[gentoo-portage-dev] [PATCH 2/3] Have repoman check that a package directory contains at least one ebuild (bug #245305).
--- bin/repoman | 8 man/repoman.1 | 3 +++ 2 files changed, 11 insertions(+) diff --git a/bin/repoman b/bin/repoman index 9b703dc..3263ceb 100755 --- a/bin/repoman +++ b/bin/repoman @@ -330,6 +330,7 @@ qahelp = { SRC_URI.mirror: A uri listed in profiles/thirdpartymirrors is found in SRC_URI, ebuild.syntax: Error generating cache entry for ebuild; typically caused by ebuild syntax error or digest verification failure, ebuild.output: A simple sourcing of the ebuild produces output; this breaks ebuild policy., + ebuild.missing: A package directory must at least contain one ebuild or be treecleaned., ebuild.nesteddie: Placing 'die' inside ( ) prints an error, but doesn't stop the ebuild., variable.invalidchar: A variable contains an invalid character that is not part of the ASCII character set, variable.readonly: Assigning a readonly variable, @@ -1466,6 +1467,13 @@ for x in effective_scanlist: can_force = False continue + if len(ebuildlist) == 0: + stats[ebuild.missing] += 1 + fails[ebuild.missing].append(%s must at least contain one % x + \ + ebuild or be treecleaned.) + can_force = False + continue + # Sort ebuilds in ascending order for the KEYWORDS.dropped check. ebuildlist = sorted(pkgs.values()) ebuildlist = [pkg.pf for pkg in ebuildlist] diff --git a/man/repoman.1 b/man/repoman.1 index e739d56..2bf3765 100644 --- a/man/repoman.1 +++ b/man/repoman.1 @@ -301,6 +301,9 @@ Ebuilds that exist but have not been added to cvs .B ebuild.output A simple sourcing of the ebuild produces output; this breaks ebuild policy. .TP +.B ebuild.missing +A package directory must at least contain one ebuild or be treecleaned. +.TP .B ebuild.patches PATCHES variable should be a bash array to ensure white space safety .TP -- 1.8.5.2
[gentoo-portage-dev] Repoman patches for bugs #205909, #245305 and #482084.
In reply, you will find three repoman patches; PATCH 1 is a bit more complex which I will detail here, the other two patches should be fairly trivial. In the first patch I need to use the @system set, as I only want to check DEPEND for packages not in the @system set; thus here is kept in mind that the @system set could possible change, in which case the check continues to work. After checking up with Arfrever, a first version that I came up with is +from portage._sets.profiles import PackagesSystemSet +system_set_atoms = PackagesSystemSet(portage.settings.profiles).getAtoms() but I am not sure if this is appropriate when used in other repositories. Arfrever warned me about this but I think I do not fully understand that. Both archive_formats* variables are based on the PMS specifications. The rest of the code has comments and should be trivial to understand. For the check name we came up with unpack.DEPEND.missing; most of the checks are two words, so, I don't know if three words is permitted. At least repoman runs without complaining or bailing out because of this. There's still a TODO left in the code that leaves me in doubt on how to properly ask the keywords to Portage; seems that I still need to learn to find my way through the documentation, but I guess after getting pointed to it a few times it will become easier. These are my first patches to the Repoman code, all three patches introduce a new warning / error, therefore it might be possible that I missed something. Grepping on an existing warning I saw the man page needs to be updated; since I never did that before, feel free to check if the syntax of that is right. Thank you for taking your time to review these. -- bin/repoman | 63 +++ man/repoman.1 | 10 ++ pym/repoman/checks.py | 10 ++ 3 files changed, 83 insertions(+) [PATCH 1/3] Have repoman check if the packages to unpack rare archive formats from SRC_URI are present in DEPEND (bug #205909). [PATCH 2/3] Have repoman check that a package directory contains at least one ebuild (bug #245305). [PATCH 3/3] Have repoman deprecate G2CONF for the GNOME team. (bug #482084). -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D
Re: [gentoo-portage-dev] [PATCH 3/3] Have repoman deprecate G2CONF for the GNOME team. (bug #482084).
On Jan 15, 2014 7:08 PM, Tom Wijsman tom...@gentoo.org wrote: --- bin/repoman | 2 ++ man/repoman.1 | 3 +++ pym/repoman/checks.py | 10 ++ 3 files changed, 15 insertions(+) diff --git a/bin/repoman b/bin/repoman index 3263ceb..6754edd 100755 --- a/bin/repoman +++ b/bin/repoman @@ -318,6 +318,7 @@ qahelp = { EAPI.incompatible: Ebuilds that use features that are only available with a different EAPI, EAPI.unsupported: Ebuilds that have an unsupported EAPI version (you must upgrade portage), SLOT.invalid: Ebuilds that have a missing or invalid SLOT variable value, + G2CONF.deprecated: G2CONF is deprecated, see Gentoo bug #482084 and the GNOME team policies, HOMEPAGE.missing: Ebuilds that have a missing or empty HOMEPAGE variable, HOMEPAGE.virtual: Virtuals that have a non-empty HOMEPAGE variable, PDEPEND.suspect: PDEPEND contains a package that usually only belongs in DEPEND., @@ -382,6 +383,7 @@ qawarnings = set(( dependency.badtilde, DESCRIPTION.toolong, EAPI.deprecated, +G2CONF.deprecated, HOMEPAGE.virtual, LICENSE.deprecated, LICENSE.virtual, diff --git a/man/repoman.1 b/man/repoman.1 index 2bf3765..7ec43d5 100644 --- a/man/repoman.1 +++ b/man/repoman.1 @@ -227,6 +227,9 @@ Syntax error in RESTRICT (usually an extra/missing space/parenthesis) .B SLOT.invalid Ebuilds that have a missing or invalid SLOT variable value .TP +.B G2CONF.deprecated +G2CONF is deprecated, see Gentoo bug #482084 and the GNOME team policies +.TP .B SRC_URI.mirror A uri listed in profiles/thirdpartymirrors is found in SRC_URI .TP diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py index 85aa065..c2608b0 100644 --- a/pym/repoman/checks.py +++ b/pym/repoman/checks.py @@ -799,6 +799,16 @@ class PortageInternalVariableAssignment(LineCheck): e += ' on line: %d' return e +class DeprecateG2CONF(LineCheck): + repoman_check_name = 'G2CONF.deprecated' + re = re.compile(r'.*G2CONF.*') + + def check(self, num, line): + Run the check on line and return error if there is one + m = self.re.match(line) + if m is not None: + return (G2CONF on line %d is deprecated, see Gentoo bug #482084.) Are you missing the line number interpolation here? or %d is supposed to be returned? + _base_check_classes = (InheritEclass, LineCheck, PhaseCheck) _constant_checks = None -- 1.8.5.2 Other than that, Looks good to me.
Re: [gentoo-portage-dev] [PATCH 1/3] Have repoman check if the packages to unpack rare archive formats from SRC_URI are present in DEPEND (bug #205909).
On Wed, Jan 15, 2014 at 4:07 PM, Tom Wijsman tom...@gentoo.org wrote: --- bin/repoman | 53 + man/repoman.1 | 4 2 files changed, 57 insertions(+) I urge you to not author new checks like this. /usr/bin/repoman is already a mess. Write these checks as functions, put them in a different file, and just call them from the giant messy loop. At least that way the checks are self contained (great for avoiding things like variable re-use or shadowing). -A diff --git a/bin/repoman b/bin/repoman index d1542e9..9b703dc 100755 --- a/bin/repoman +++ b/bin/repoman @@ -36,6 +36,9 @@ pym_path = osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), pym) sys.path.insert(0, pym_path) import portage portage._internal_caller = True + +from portage._sets.profiles import PackagesSystemSet +system_set_atoms = PackagesSystemSet(portage.settings.profiles).getAtoms() portage._disable_legacy_globals() try: @@ -300,6 +303,7 @@ qahelp = { inherit.missing: Ebuild uses functions from an eclass but does not inherit it, inherit.unused: Ebuild inherits an eclass but does not use it, java.eclassesnotused: With virtual/jdk in DEPEND you must inherit a java eclass, + unpack.DEPEND.missing: A rare archive format was used in SRC_URI, but its package to unpack it is missing in DEPEND., wxwidgets.eclassnotused: Ebuild DEPENDs on x11-libs/wxGTK without inheriting wxwidgets.eclass, KEYWORDS.dropped: Ebuilds that appear to have dropped KEYWORDS for some arch, KEYWORDS.missing: Ebuilds that have a missing or empty KEYWORDS variable, @@ -399,6 +403,7 @@ qawarnings = set(( metadata.warning, portage.internal, repo.eapi.deprecated, +unpack.DEPEND.missing, usage.obsolete, upstream.workaround, LIVEVCS.stable, @@ -479,6 +484,25 @@ ruby_deprecated = frozenset([ ruby_targets_ree18, ]) +# TODO: Add functionality to support checking for deb2targz on platforms where +# GNU binutils is absent; see PMS 5, section 11.3.3.13. +archive_formats = { + \.7[zZ]:app-arch/p7zip, + \.(bz2?|tbz2):app-arch/bzip2, + \.jar:app-arch/unzip, + \.(LH[aA]|lha|lzh):app-arch/lha, + \.lzma:app-arch/lzma-utils, + \.(rar|RAR):app-arch/unrar, + \.(tar(\.(bz2?|gz|Z))?|tbz2|t[bg]z)?:app-arch/tar, + \.(gz|tar\.Z|t[bg]z|[zZ]):app-arch/gzip, + \.(zip|ZIP):app-arch/unzip, +} + +archive_formats_eapi_3_to_5 = { + \.tar.xz:app-arch/tar, + \.xz:app-arch/xz-utils, +} + metadata_xml_encoding = 'UTF-8' metadata_xml_declaration = '?xml version=1.0 encoding=%s?' % \ (metadata_xml_encoding,) @@ -1559,6 +1583,7 @@ for x in effective_scanlist: fetchlist_dict = portage.FetchlistDict(checkdir, repoman_settings, portdb) myfiles_all = [] src_uri_error = False + needed_unpack_depends = {} for mykey in fetchlist_dict: try: myfiles_all.extend(fetchlist_dict[mykey]) @@ -1573,7 +1598,22 @@ for x in effective_scanlist: stats[SRC_URI.syntax] += 1 fails[SRC_URI.syntax].append( %s.ebuild SRC_URI: %s % (mykey, e)) + + # Compare each SRC_URI entry against archive_formats; if one of the + # extensions match, we remember which archive depends are needed to + # check them later on. + needed_unpack_depends[mykey] = [] + for file_extension in archive_formats or \ + ((re.match('[345]$', eapi) is not None) \ + and file_extension in archive_formats_eapi_3_to_5): + for entry in fetchlist_dict[mykey]: + if re.match('.*%s$' % file_extension, entry) is not None: + format = archive_formats[file_extension] + + if format not in needed_unpack_depends[mykey]: + needed_unpack_depends[mykey].append(format) del fetchlist_dict + if not src_uri_error: # This test can produce false positives if SRC_URI could not # be parsed for one or more ebuilds. There's no point in @@ -2010,6 +2050,17 @@ for x in effective_scanlist: atoms = None badsyntax.append(str(e)) + if atoms and mytype == 'DEPEND': + # We check whether the needed archive dependencies are present + # in DEPEND, which were determined from SRC_URI. + for entry in needed_unpack_depends[catdir + '/' + y]: + if entry not in system_set_atoms and entry \ +
Re: [gentoo-portage-dev] [PATCH 1/3] Have repoman check if the packages to unpack rare archive formats from SRC_URI are present in DEPEND (bug #205909).
Am 16.01.2014 01:07, schrieb Tom Wijsman: --- bin/repoman | 53 + man/repoman.1 | 4 2 files changed, 57 insertions(+) diff --git a/bin/repoman b/bin/repoman index d1542e9..9b703dc 100755 --- a/bin/repoman +++ b/bin/repoman @@ -36,6 +36,9 @@ pym_path = osp.join(osp.dirname(osp.dirname(osp.realpath(__file__))), pym) sys.path.insert(0, pym_path) import portage portage._internal_caller = True + +from portage._sets.profiles import PackagesSystemSet +system_set_atoms = PackagesSystemSet(portage.settings.profiles).getAtoms() portage._disable_legacy_globals() You should be using repoman_settings instead of portage.settings. Considering the later use, you don't need PackagesSystemSet set here, just use a set. And use atom.cp instead of the atoms. try: @@ -300,6 +303,7 @@ qahelp = { inherit.missing: Ebuild uses functions from an eclass but does not inherit it, inherit.unused: Ebuild inherits an eclass but does not use it, java.eclassesnotused: With virtual/jdk in DEPEND you must inherit a java eclass, + unpack.DEPEND.missing: A rare archive format was used in SRC_URI, but its package to unpack it is missing in DEPEND., wxwidgets.eclassnotused: Ebuild DEPENDs on x11-libs/wxGTK without inheriting wxwidgets.eclass, KEYWORDS.dropped: Ebuilds that appear to have dropped KEYWORDS for some arch, KEYWORDS.missing: Ebuilds that have a missing or empty KEYWORDS variable, @@ -399,6 +403,7 @@ qawarnings = set(( metadata.warning, portage.internal, repo.eapi.deprecated, +unpack.DEPEND.missing, usage.obsolete, upstream.workaround, LIVEVCS.stable, @@ -479,6 +484,25 @@ ruby_deprecated = frozenset([ ruby_targets_ree18, ]) +# TODO: Add functionality to support checking for deb2targz on platforms where +# GNU binutils is absent; see PMS 5, section 11.3.3.13. +archive_formats = { + \.7[zZ]:app-arch/p7zip, + \.(bz2?|tbz2):app-arch/bzip2, + \.jar:app-arch/unzip, + \.(LH[aA]|lha|lzh):app-arch/lha, + \.lzma:app-arch/lzma-utils, + \.(rar|RAR):app-arch/unrar, + \.(tar(\.(bz2?|gz|Z))?|tbz2|t[bg]z)?:app-arch/tar, + \.(gz|tar\.Z|t[bg]z|[zZ]):app-arch/gzip, + \.(zip|ZIP):app-arch/unzip, +} + +archive_formats_eapi_3_to_5 = { + \.tar.xz:app-arch/tar, + \.xz:app-arch/xz-utils, +} + metadata_xml_encoding = 'UTF-8' metadata_xml_declaration = '?xml version=1.0 encoding=%s?' % \ (metadata_xml_encoding,) @@ -1559,6 +1583,7 @@ for x in effective_scanlist: fetchlist_dict = portage.FetchlistDict(checkdir, repoman_settings, portdb) myfiles_all = [] src_uri_error = False + needed_unpack_depends = {} for mykey in fetchlist_dict: try: myfiles_all.extend(fetchlist_dict[mykey]) @@ -1573,7 +1598,22 @@ for x in effective_scanlist: stats[SRC_URI.syntax] += 1 fails[SRC_URI.syntax].append( %s.ebuild SRC_URI: %s % (mykey, e)) + + # Compare each SRC_URI entry against archive_formats; if one of the + # extensions match, we remember which archive depends are needed to + # check them later on. + needed_unpack_depends[mykey] = [] + for file_extension in archive_formats or \ + ((re.match('[345]$', eapi) is not None) \ Use portage.eapi for the line above. You may have to add a new function to portage.eapi. + and file_extension in archive_formats_eapi_3_to_5): + for entry in fetchlist_dict[mykey]: + if re.match('.*%s$' % file_extension, entry) is not None: + format = archive_formats[file_extension] As these regex are used frequently, they should be compiled using re.compile. + + if format not in needed_unpack_depends[mykey]: + needed_unpack_depends[mykey].append(format) I'd make needed_unpack_depends[mykey] a set. Then you can just add() instead of checking and appending. del fetchlist_dict + if not src_uri_error: # This test can produce false positives if SRC_URI could not # be parsed for one or more ebuilds. There's no point in @@ -2010,6 +2050,17 @@ for x in effective_scanlist: atoms = None badsyntax.append(str(e)) + if atoms and mytype == 'DEPEND': Use if atoms and buildtime: here. + # We check whether the needed archive dependencies are present + # in DEPEND, which were determined from SRC_URI. + for entry in