Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On Mon, 15 Sep 2014, Rich Freeman wrote: I'll add this to the next Council agenda. I think this is ripe for discussion. The last discussion of this really wasn't aimed at git anyway. Some of the arguments back then were that a) ChangeLogs are aimed at users, so they don't necessarily contain the same information as the commit log (i.e. could be seen as NEWS files) and b) that it should be possible to edit them, for example, to correct typos or wrong bug# references. I fail to see how anything of this would depend on the underlying VCS used. Ulrich pgpVnJdUsiCfH.pgp Description: PGP signature
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
Ulrich Mueller: ChangeLogs are aimed at users Did any1 ask them if they care?
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On 16/09/2014 12:18, hasufell wrote: Ulrich Mueller: ChangeLogs are aimed at users Did any1 ask them if they care? I'm a user and I don't care. I use diff. I only go to the Changelog when I can't determine the maintainers intent from diff and the ebuild content. That happens maybe once a year. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On Tue, Sep 16, 2014 at 6:18 AM, hasufell hasuf...@gentoo.org wrote: Ulrich Mueller: ChangeLogs are aimed at users Did any1 ask them if they care? I'm sure somebody will reply and say that they care. It still seems like a lot of overhead to me for a very one-off workflow. Maybe if portage automatically output the relevant changelog entries in pretend mode we could pretend that they're news or something like that. Most likely, if you stick something important in the changelog it will be read by maybe 0.1% of our users before emerging the package. Maybe if you're lucky 20% of people running into some kind of breakage will read the changelog after the fact. I imagine that 19.5% of those 20% would check the git log if the changelog didn't exist. If we actually move to a model where many users actually sync their trees from git, then I'd expect the changelogs to be even less useful. After all, git will actually tell you what changed since your last sync. -- Rich
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
Rich Freeman: On Tue, Sep 16, 2014 at 6:18 AM, hasufell hasuf...@gentoo.org wrote: Ulrich Mueller: ChangeLogs are aimed at users Did any1 ask them if they care? I'm sure somebody will reply and say that they care. It still seems like a lot of overhead to me for a very one-off workflow. Maybe if portage automatically output the relevant changelog entries in pretend mode we could pretend that they're news or something like that. Most likely, if you stick something important in the changelog it will be read by maybe 0.1% of our users before emerging the package. Maybe if you're lucky 20% of people running into some kind of breakage will read the changelog after the fact. I imagine that 19.5% of those 20% would check the git log if the changelog didn't exist. If we actually move to a model where many users actually sync their trees from git, then I'd expect the changelogs to be even less useful. After all, git will actually tell you what changed since your last sync. And git allows you to _properly_ check for changes, because all changes are in one history, so you don't have to grep 3+ ChangeLogs (e.g. in eclasses, profiles and licenses) in order to know what happened. Even easier... related changes might just go in one commit and when you look for it you'll also see the other files that have been modified as part of a version bump (e.g. a useflag mask or whatever). The only place I actually look for changes is the gentoo-commits ML which is kind of the poor version of a git history.
[gentoo-dev] Re: [gentoo-scm] Re: My masterplan for git migration (+ looking for infra to test it)
On Mon, Sep 15, 2014 at 11:50 PM, Brian Harring ferri...@gmail.com wrote: On Sun, Sep 14, 2014 at 10:33 AM, Rich Freeman ri...@gentoo.org wrote: On Sun, Sep 14, 2014 at 8:03 AM, Michał Górny mgo...@gentoo.org wrote: Of course, that assumes infra is going to cooperate quickly or someone else is willing to provide the infra for it. The infra components to a git infrastructure are one of the main blockers at this point. I don't really see cooperation as the issue - just lack of manpower or interest. I can't speak for current gentoo infra, but it'd help to think through exactly who will have RO access to the git repo. Trying to have the entire gentoo userbase accessing the repo via git is going to be resource intensive- if you're intending that, well, start donating frankly. If however you're angling for just devs having git access, and utilizing the existing rsync mirror, that should be a helluva lot more doable (even with shitty hardware). I believe that is the proposal here. There would be a dev-only main repository which is where all commits would go. Then there would be a power-user sync repo derived from that, and then an rsync tree derived from that. This isn't all that different from what we do with cvs. Git is pretty easy to mirror, since it is after all a distributed system at its heart. It would probably make sense to do one push/pull from the main repository to the power-user repo, do whatever magic needs to be done with it, and then push out from there to a mirror network, github, wherever. We'd probably want mirrors of both the original dev tree and the one with metadata and such. People love to point out linux and its insane commit rate. The thing is, the mainline git repo with all those commits has exactly one committer - Linus himself. They don't have one big repo with one master branch that everybody pushes to. At least, that is my understanding (and there are certainly others here who are more involved with kernel development). To be frank, I think you're seriously overthinking it and worrying about a nonissue. Worrying about this while ignoring security enhancements like requiring signed commits on push is a bit weird to me. So far the discussion in the thread has included the commit signing issue. I tend to agree that commit rate probably won't be a problem after some discussion, as long as we don't require a repoman check 100% of the time in-between pull and push. For a single package that won't be a problem, but doing it at the category/tree level is just going to make collisions inevitable. One additional point no one has mentioned; when cutting over to git, afterwards someone will need to go through and rip out all of the cvs $Id style tags that exist in the tree- depending on how folks are doing the conversion, it might be worth just jamming that in right off the bat rather than trying to clean the tree afterwards. I'd suggest that we not go too much further with editing history - consolidating Manifest/package commits did make sense, and the other fixes do as well. I'd prefer not to go trying to remove keywords from the tree during conversion. We've already had all kinds of headaches from keywords as it is (especially with patches/etc). I suggest that we initially convert the tree and leave the keywords in, and we can always clean them up later, either by script or manually. I'd suggest keeping scripting to a minimum guaranteed-safe level (such as just our ebuild headers), and leave it up to maintainers to get any stray ones. In-band signaling is lousy design in this day and age. Oh, did somebody bring up Changelogs? :) ~ferringb, who was enjoying his retirement very much thank you I appreciate your email - there is a lot of history we can stand to benefit from. I think I have a container set up now that can efficiently migrate cvs-git using your scripts, and I'm just doing some more testing and work to make it fully transportable. From there we just need to figure out where to run it when the time comes, which should be the easy part. I do believe that mgorny was offering to step up to help out with the scripting though. If we have the scripts/hooks needed to manipulate gentoo-x86 to produce the various trees, then I think that transplanting them to infra will not be a difficult next step. Really that part is the part which is most lacking right now, so if a few are willing to step up on this I think we can get somewhere. -- Rich
Re: [gentoo-dev] git security (SHA-1)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 15/09/14 07:59 PM, Rich Freeman wrote: On Mon, Sep 15, 2014 at 6:11 PM, Gordon Pettey petteyg...@gmail.com wrote: Even if you wanted to burn the money to find that magical collision that actually contains working code, you've still got to somehow propagate that to other repositories, since they'll just ignore it for having the same hash as an already-existing object. Well, if you're willing to trust that nobody is able to tamper with repositories, then you don't need gpg signatures in the first place. I think that gpg signatures protected by an SHA1 hash provide fairly little security - a chain is as strong as its weakest link and sha1 has been considered fairly weak for years now. However, I think it does make sense to at least get gpg into the workflow in the hopes that some day git will move to a stronger hash, and since it isn't a huge hardship to do so. I wouldn't make too light of the use of SHA1 though. As you point out simply exploiting it isn't enough, but the whole reason for having signatures is to make an attack on a central repository useless. Having gpg on top of ssh keys and all that is obviously redundant, but that is the whole point of it. -- Rich If the issue preventing protection is that the gpg signature only signs the hash, couldn't we just make repoman automatically add to the bottom of the comment a clearsign on the contents of the commit? -BEGIN PGP SIGNATURE- Version: GnuPG v2 iF4EAREIAAYFAlQYPskACgkQ2ugaI38ACPDjowEAmfMQePUgmLSDrmKyXxdUfbil g6KVaPkL1yfDwrLP7J8BAK+g5MMCMDgH9wDzEHIYerDi9ZIm39AfwazQF3mz3dPR =slAr -END PGP SIGNATURE-
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
El mar, 16-09-2014 a las 07:26 -0400, Rich Freeman escribió: On Tue, Sep 16, 2014 at 6:18 AM, hasufell hasuf...@gentoo.org wrote: Ulrich Mueller: ChangeLogs are aimed at users Did any1 ask them if they care? I'm sure somebody will reply and say that they care. It still seems like a lot of overhead to me for a very one-off workflow. Maybe if portage automatically output the relevant changelog entries in pretend mode we could pretend that they're news or something like that. Most likely, if you stick something important in the changelog it will be read by maybe 0.1% of our users before emerging the package. Maybe if you're lucky 20% of people running into some kind of breakage will read the changelog after the fact. I imagine that 19.5% of those 20% would check the git log if the changelog didn't exist. If we actually move to a model where many users actually sync their trees from git, then I'd expect the changelogs to be even less useful. After all, git will actually tell you what changed since your last sync. -- Rich Maybe one option would be to kill Changelogs and provide a script to let people get git messages and reformat them in a way similar as current ChangeLog files, that way people will still be able to save this information for the future (if they won't have internet conection later for example) and read it simply with less for example. With this option, we won't need to provide Changelogs and distribute them but people wanting to have them will still be able to generate them if wanted (for example, just after updating portage tree)
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On Tue, Sep 16, 2014 at 9:44 AM, Pacho Ramos pa...@gentoo.org wrote: Maybe one option would be to kill Changelogs and provide a script to let people get git messages and reformat them in a way similar as current ChangeLog files, that way people will still be able to save this information for the future (if they won't have internet conection later for example) and read it simply with less for example. With this option, we won't need to provide Changelogs and distribute them but people wanting to have them will still be able to generate them if wanted (for example, just after updating portage tree) Or they could just clone the git tree, and they can look at per-file logs anytime they want to. I mean, sure, we COULD do this stuff. But, why? It isn't like kernel.org has some tool that lets kernel users generate per-file changelog histories just in case they don't want to use git. If somebody wants to build a tool like this by all means go ahead and do it. I just don't see it as something that should be a migration pre-requisite. That's just my opinion though. -- Rich
Re: [gentoo-dev] git security (SHA-1)
On Tue, Sep 16, 2014 at 9:44 AM, Ian Stakenvicius a...@gentoo.org wrote: If the issue preventing protection is that the gpg signature only signs the hash, couldn't we just make repoman automatically add to the bottom of the comment a clearsign on the contents of the commit? The gpg signature is on the entire contents of the commit. However, the contents of the commit do not include the files that are being committed - it includes hashes of the parent commit, the commit message, other headers, and the hash of the tree being committed, which is sha1. That last hash is the only thing that ties the commit to the files being committed, so you can modify the files all you like as long as the sha1 is the same. I don't think we should try to fix git. It makes far more sense to have upstream fix it. I don't think we should hold up the migration over it - NOBODY is holding off on adopting git over this stuff and I'm not even aware of any projects that gpg sign their git commits. Remember, the data model is: commit -- tree -- [tree...] -- blob The signature is against the commit, and sha1 hashes are what tie everything else to it. If you want to satisfy yourself I believe you can get git to dump the contents of any object without formatting/etc. You'll see that the gpg signature matches the content of the commit (minus the gpg signature header, of course). If you directly access objects from the filesystem I think git prepends a hash or something to the start of every file. -- Rich
Re: [gentoo-dev] git security (SHA-1)
On 09/16/2014 10:03 AM, Rich Freeman wrote: The gpg signature is on the entire contents of the commit. However, the contents of the commit do not include the files that are being committed - it includes hashes of the parent commit, the commit message, other headers, and the hash of the tree being committed, which is sha1. That last hash is the only thing that ties the commit to the files being committed, so you can modify the files all you like as long as the sha1 is the same. To put things in perspective, all I had to do was ask for commit access and somebody eventually gave it to me. We should worry about this when breaking SHA1 becomes less expensive than the ebuild quizzes.
Re: [gentoo-dev] git security (SHA-1)
On 17 September 2014 01:44, Ian Stakenvicius a...@gentoo.org wrote: bottom of the comment a clearsign on the contents of the commit? I don't see that being useful as written, because that's presently exactly what git does. the very best I think you could do is sign the commit *diff*, ie: recursively compare TREE and PARENT-TREE and feed only *new* objects through GPG. That's the most meaningful thing you'll get. ( And it really is up-streams responsibility to implement this feature if they care, bolting it on after the fact is horrible ) -- Kent *KENTNL* - https://metacpan.org/author/KENTNL
Re: [gentoo-dev] git security (SHA-1)
Michael Orlitzky: On 09/16/2014 10:03 AM, Rich Freeman wrote: The gpg signature is on the entire contents of the commit. However, the contents of the commit do not include the files that are being committed - it includes hashes of the parent commit, the commit message, other headers, and the hash of the tree being committed, which is sha1. That last hash is the only thing that ties the commit to the files being committed, so you can modify the files all you like as long as the sha1 is the same. To put things in perspective, all I had to do was ask for commit access and somebody eventually gave it to me. We should worry about this when breaking SHA1 becomes less expensive than the ebuild quizzes. Yep, that's what I'd try to do actually if I was working for NSA (uh-oh). Try to get collaborators into every possible opensource project. There are so many thing you can do... e.g. fix a security bug, but reference a self-packaged tarball from your dev space (which still contains the exploit) in the ebuild. No one will know. And that's a pretty low hanging fruit.
Re: [gentoo-dev] git security (SHA-1)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 16/09/14 10:33 AM, Kent Fredric wrote: On 17 September 2014 01:44, Ian Stakenvicius a...@gentoo.org mailto:a...@gentoo.org wrote: bottom of the comment a clearsign on the contents of the commit? [...] the very best I think you could do is sign the commit *diff*, Yes, that's what I meant by the contents of the commit. Apologies for not using proper terminology. -BEGIN PGP SIGNATURE- Version: GnuPG v2 iF4EAREIAAYFAlQYTIMACgkQ2ugaI38ACPBQzQD8Cc/7i/Q7v0NVv0duVNkpDAER 1PDOT+TLIDbv9DG0YW4BAIGSSLb0dahPp6+bdbe9AUR63m9QIrw51TqL3lGpGsm8 =jEVx -END PGP SIGNATURE-
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
El mar, 16-09-2014 a las 09:55 -0400, Rich Freeman escribió: On Tue, Sep 16, 2014 at 9:44 AM, Pacho Ramos pa...@gentoo.org wrote: Maybe one option would be to kill Changelogs and provide a script to let people get git messages and reformat them in a way similar as current ChangeLog files, that way people will still be able to save this information for the future (if they won't have internet conection later for example) and read it simply with less for example. With this option, we won't need to provide Changelogs and distribute them but people wanting to have them will still be able to generate them if wanted (for example, just after updating portage tree) Or they could just clone the git tree, and they can look at per-file logs anytime they want to. I mean, sure, we COULD do this stuff. But, why? It isn't like kernel.org has some tool that lets kernel users generate per-file changelog histories just in case they don't want to use git. If somebody wants to build a tool like this by all means go ahead and do it. I just don't see it as something that should be a migration pre-requisite. That's just my opinion though. -- Rich I don't consider it a pre-requisite either, was only trying to give an option to still tell people how to get a ChangeLog similar to current ones easily (as looks like they are used a lot per the past discussion :/) I remember something similar was done in the past when gnome stuff moved to git: https://wiki.gnome.org/Git/ChangeLog But I guess once we get habituated to simply review something equivalent to https://git.gnome.org/browse/ not many people will miss the old Changelogs ;)
Re: [gentoo-dev] git security (SHA-1)
Rich Freeman wrote: If you want to satisfy yourself I believe you can get git to dump the contents of any object without formatting/etc. git ls-tree HEAD . git show $blobhash git show --pretty=raw HEAD //Peter
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
On 14/09/14 17:15, Kent Fredric wrote: On 15 September 2014 02:40, Michał Górny mgo...@gentoo.org wrote: However, I'm wondering if it would be possible to restrict people from accidentally committing straight into github (e.g. merging pull requests there instead of to our main server). Easy. Put the Gentoo repo in its own group. Don't give anyone any kinds of permissions on it. Have only one approved account for the purpose of pushing commits. Have a post-push hook that replicates to github as that approved account = Github is just a read only mirror, any pull reqs submitted there will be fielded and pushed to gentoo directly. Only downside there is the way github pull reqs work is if the final SHA1's that hit tree don't match, the pull req doesn't close. Solutions: - A) Have somebody tasked with reaping old pull reqs with permissions granted. ( Uck ) - B) Always use a merge of some kind to mark the pull req as dead ( for instance, an ours merge to mark the branch as deprecated ) C) Ask nicely Github to have an application key and have a pull-request bridge to avoid the problem completely. I'd complete the migration first and discuss this kind of details later. lu
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
On 14/09/14 16:46, Michał Górny wrote: Of course, if we can't spare the resources to do intermediate updates, we may as well switch to cron-based update method. The mirror have a sync time, so basically regenerating the cache and pushing the tree with further toward the user can happen the same way w/out impacting anybody. We could just snapshot the tree when the regen starts and push that commit to the user git and rsync. People is still supposed to play nice and sync not every minute. lu
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
On 14/09/14 17:30, Patrick Lauer wrote: Are we going to disallow merge commits and ask devs to rebase local changes in order to keep the history clean? Is that going to be sane with our commit frequency? Which is our commit frequency? Worst case we can aggregate changes and push them in bulk. lu
[gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
Anthony G. Basile posted on Mon, 15 Sep 2014 17:43:09 -0400 as excerpted: We could just push out the word that ChangeLogs are going away and they have to read the git repo. That might be the easiest solution. I do have users that quote my ChangeLogs though. As such a user... Given the proposed three-level system, dev-git, power-user-git, general- user-rsync, dumping changelogs in rsync is my preferred solution as well. Given power-user-git access I expect anyone who actually reads changelogs will be switching to the power-user-git level in a heartbeat, and I can't see the folks remaining on rsync actually caring about changelogs, either. In the one-off case they find themselves needing a changelog, they can read the git commit log online. When the discussion came up previously I was strongly in favor of keeping changelogs, because users /don't/ currently have a reasonable alternative. With the proposed three-level system, the power-user git repo will be that alternative, and the changelogs can simply go away in favor of people actually having access to a full git repo and git log. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
On 15/09/14 01:21, Patrick Lauer wrote: On Sunday 14 September 2014 15:42:15 hasufell wrote: Patrick Lauer: Are we going to disallow merge commits and ask devs to rebase local changes in order to keep the history clean? Is that going to be sane with our commit frequency? You have to merge or rebase anyway in case of a push conflict, so the only difference is the method and the effect on the history. Currently... CVS allows you to run repoman on an outdated tree and push broken ebuilds with repoman being happy. Git will not allow this. iow, git doesn't allow people to work on more than one item at a time? It does. That'd mean I need half a dozen checkouts just to emulate cvs, which somehow doesn't make much sense to me ... Your statement sounds strange to me. commands you need to know: git rebase -i git add (-p) git commit (-p) git branch/checkout Examples edit cat/pkg/foo.ebuild edit cat2/pkg/bar.ebuild edit profile git add -p# to select by line what you want in the commit git commit# and you write down the commit message git commit -p # to do both at the same time. git commit -p # again to lump other changes line by line OR edit cat/pkg/foo.ebuild git commit -a # everything (that's tracked) you edited gets in a commit edit cat/pkg/bar.ebuild git commit -a # everything (that's tracked) you edited gets in again ... git rebase -i # sort out what you want commit merge, edit, drop etc git push. Git let you do whatever you do in cvs, but in a _much_ saner and faster way.
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
On Tue, Sep 16, 2014 at 1:07 PM, Luca Barbato lu_z...@gentoo.org wrote: On 14/09/14 17:30, Patrick Lauer wrote: Are we going to disallow merge commits and ask devs to rebase local changes in order to keep the history clean? Is that going to be sane with our commit frequency? Which is our commit frequency? Worst case we can aggregate changes and push them in bulk. I don't think commit frequency will be a problem other than maybe causing repoman issues if you're doing tree-wide changes (since repoman takes a while to run). See: https://github.com/rich0/gentoo-gitmig-2014-09-15/graphs/commit-activity Our tree gets 50-150 commits/day on average it seems. I have no idea how far back the punchcard view goes, but that should give a relative sense of distribution. -- Rich
[gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
W. Trevor King posted on Mon, 15 Sep 2014 13:33:46 -0700 as excerpted: On Mon, Sep 15, 2014 at 01:29:44PM -0700, W. Trevor King wrote: I don't see any benefit to using rsync vs. a shallow clone as the transmission protocol. Other than the fact that before you dropped it you'd need to push a ‘emerge sync’ that could handle either rsync or Git, stabilize that Portage, and then wait for folks to adopt it. Portage already handles it. =:^) Quoting from the emerge (1) manpage (listed as v2.2.12, dated Mar 2014): --sync Updates repositories, for which sync-type and sync-uri attributes are set in repos.conf. See portage(5) for more information. Then in the portage (5) manpage: repos.conf Specifies site-specific repository configuration information. Configuration specified in repos.conf can be overriden by PORTAGE_REPOSITORIES environmental variable, which has the same format as repos.conf. [...] Attributes supported in sections of repositories: [...] sync-type Specifies type of synchronization performed by `emerge --sync`. Valid non-empty values: cvs, git, rsync This attribute can be set to empty value to disable synchronization of given repository. Empty value is default. sync-uri Specifies URI of repository used for synchronization performed by `emerge --sync`. This attribute can be set to empty value to disable synchronization of given repository. Empty value is default. Syntax: cvs: [cvs://]:access_method:[username@]hostname[:port]:/path git: (git|git+ssh|http|https)://[username@]hostname[:port]/path rsync: (rsync|ssh)://[username@]hostname[:port]/(module|path) So portage already handles it. =:^) Tho it's possible there are still bugs as surely that doesn't have the extremely broad long-term testing that rsync does, and I'm ~arch of course and don't know if that's stable, yet. But it's there. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On Tue, Sep 16, 2014 at 05:35:08PM +, Duncan wrote: W. Trevor King posted on Mon, 15 Sep 2014 13:33:46 -0700 as excerpted: On Mon, Sep 15, 2014 at 01:29:44PM -0700, W. Trevor King wrote: I don't see any benefit to using rsync vs. a shallow clone as the transmission protocol. Other than the fact that before you dropped it you'd need to push a ‘emerge sync’ that could handle either rsync or Git, stabilize that Portage, and then wait for folks to adopt it. Portage already handles it. =:^) Oh, lovely :). Looks like that landed in 2.2.0 with 47e8d22d (Add support for multiple repositories in `emerge --sync`, 2013-07-23). There are older Portages in the tree though (back to 2.1.6.7_p1), so you'd still want to wait until those were gone before dropping rsync. Also, I don't see a way to say “use Git to sync, but keep a shallow repository”. Ideally, we'd want: $ git clone --depth=1 git://git.gentoo.org/gentoo-portage.git for the initial clone (modulo whatever URI), and: $ git pull --depth=1 for subsequent syncs. pym/_emerge/actions.py currently hardcodes ‘git pull’ for the latter, and doesn't seem to have any code for the former. On the other hand, it wouldn't be too terrible to force users to shallow their history manually whenever they felt like it. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
[gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
Rich Freeman posted on Tue, 16 Sep 2014 09:55:31 -0400 as excerpted: Or they could just clone the git tree, and they can look at per-file logs anytime they want to. Give me ro access to a current git repo and I'll *VERY* happily leave changelogs to history along with 8-track tapes and 5.25-inch floppies! =:^) I was strongly in favor of keeping changelogs (and mandating proper add/ change/deletion entries) the last time the topic came up, but that was in the context of (web)?rsync being the only viable user sync method and thus changelogs being the only user-local-accessible record. With user- git-repo access, I'll /very/ (very, very, very...) happily leave rsync behind for git, and changelogs along with it! =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-dev] Re: RFC: kde5 and kde5-functions eclass
On 09/16/2014 02:19 AM, Davide Pesavento wrote: if [[ -a CMakeLists.txt ]]; then Unnecessary quoting. Also, -e is more common than -a I guess both the eclasses (and a lot of Gentoo stuff in general) has quoting that's not strictly necessary. Thanks for the review, everything else has been actioned.
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On Tue, Sep 16, 2014 at 10:52:13AM -0700, W. Trevor King wrote: Oh, lovely :). Looks like that landed in 2.2.0 with 47e8d22d (Add support for multiple repositories in `emerge --sync`, 2013-07-23). Actually, ‘git pull’ support in one form or another dates back to ba797c11 (Add --sync support for `git pull`, 2008-12-11), which landed in v2.2_rc18. There are older Portages in the tree though (back to 2.1.6.7_p1), so you'd still want to wait until those were gone before dropping rsync. The ‘git pull’ support was also backported to the 2.1.6.7_p1 series with d3c42937 (Add --sync support for `git pull`, 2008-12-11), which landed in v2.1.6.1, so I doubt any Portage users lack pull support. I'm not sure about folks using other package managers though. Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
[gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
W. Trevor King posted on Tue, 16 Sep 2014 10:52:13 -0700 as excerpted: Also, I don't see a way to say “use Git to sync, but keep a shallow repository”. Presumably at some point we'd get the PORTAGE_GIT* equivalent of the PORTAGE_RSYNC* settings from make.conf, but settable in both make.conf for global git-type settings and repos.conf for individual repo settings. The big step, basic git support, is already there, but it doesn't have the broad usage of rsync and thus people haven't yet run into all the corner-cases that triggered the development of all those rich rsync option settings we have access to these days. Tho I don't think it should be a blocker to the git migration, as I guess it'll be pretty mandatory that we keep an rsync backport for at least the gentoo-standard year of support. Immediately dropping stuff like changelogs from that rsync backport should make that simpler. If people want them, they can arrange to use the user-git repo. On the other hand, it wouldn't be too terrible to force users to shallow their history manually whenever they felt like it. True. Which brings up another point. A user-focused git migration guide document would be very nice and very gentoo. =:^) Once we have the git thing up and running for the early-adopters who can more or less find their own way, I'd call that a blocker to the possibility of eventually turning off rsync entirely, for our stable users. Again, *NOT* a blocker to turning on git for those who can find their own way, but to eventually turning off the rsync backward compatibility machinery for stable. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
On Tue, 16 Sep 2014 10:52:13 -0700 W. Trevor King wk...@tremily.us wrote: On Tue, Sep 16, 2014 at 05:35:08PM +, Duncan wrote: W. Trevor King posted on Mon, 15 Sep 2014 13:33:46 -0700 as excerpted: On Mon, Sep 15, 2014 at 01:29:44PM -0700, W. Trevor King wrote: I don't see any benefit to using rsync vs. a shallow clone as the transmission protocol. Other than the fact that before you dropped it you'd need to push a ‘emerge sync’ that could handle either rsync or Git, stabilize that Portage, and then wait for folks to adopt it. Portage already handles it. =:^) Oh, lovely :). Looks like that landed in 2.2.0 with 47e8d22d (Add support for multiple repositories in `emerge --sync`, 2013-07-23). There are older Portages in the tree though (back to 2.1.6.7_p1), so you'd still want to wait until those were gone before dropping rsync. Also, I don't see a way to say “use Git to sync, but keep a shallow repository”. Ideally, we'd want: $ git clone --depth=1 git://git.gentoo.org/gentoo-portage.git for the initial clone (modulo whatever URI), and: $ git pull --depth=1 for subsequent syncs. pym/_emerge/actions.py currently hardcodes ‘git pull’ for the latter, and doesn't seem to have any code for the former. On the other hand, it wouldn't be too terrible to force users to shallow their history manually whenever they felt like it. Cheers, Trevor The depth option will be added to the new portage plugin-sync system in final stages of development now. There will be 2 new repos.conf options 1 for new repo install eg. git clone: --depth=1 another for sync options: eg. git pull: --rebase origin master this will allow changes to a repo in a different branch be updated from the master branch. they will be repo specific options, not global. The new system will allow emerge-webrync type repos to be synced via emerge --sync instead of the emerge-webrsync command. Plus it will add an svn type and the ability for a layman type which layman already has code for. Layman is just waiting for the new sync system to land in portage's master branch before enabling it to be installed. It will also allow for third party sync modules to be created and easily installed. So a squashfs sync type could be created and installed for those repos where that a squashfs tree is offered. -- Brian Dolbec dolsen signature.asc Description: PGP signature
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
Il 16/09/2014 20:02, Duncan ha scritto: Rich Freeman posted on Tue, 16 Sep 2014 09:55:31 -0400 as excerpted: Or they could just clone the git tree, and they can look at per-file logs anytime they want to. Give me ro access to a current git repo and I'll *VERY* happily leave changelogs to history along with 8-track tapes and 5.25-inch floppies! =:^) I was strongly in favor of keeping changelogs (and mandating proper add/ change/deletion entries) the last time the topic came up, but that was in the context of (web)?rsync being the only viable user sync method and thus changelogs being the only user-local-accessible record. With user- git-repo access, I'll /very/ (very, very, very...) happily leave rsync behind for git, and changelogs along with it! =:^) yes, this probably is the same for everyone, and if it's not it should be anyway.
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
Dnia 2014-09-16, o godz. 19:05:18 Luca Barbato lu_z...@gentoo.org napisał(a): On 14/09/14 16:46, Michał Górny wrote: Of course, if we can't spare the resources to do intermediate updates, we may as well switch to cron-based update method. The mirror have a sync time, so basically regenerating the cache and pushing the tree with further toward the user can happen the same way w/out impacting anybody. We could just snapshot the tree when the regen starts and push that commit to the user git and rsync. People is still supposed to play nice and sync not every minute. People don't have to sync. They will pull, and pulling often doesn't really hurt servers like rsync does. Of course, I'm considering the users switching to git there. However, I don't think limitations of rsync should impact them. -- Best regards, Michał Górny signature.asc Description: PGP signature
[gentoo-dev] Add bc back to the stage3
The bc utility is part of the posix tools and it might be used to build linux among the other stuff. lu
Re: [gentoo-dev] My masterplan for git migration (+ looking for infra to test it)
On 09/16/2014 01:56 PM, hasufell wrote: Luca Barbato: On 15/09/14 01:21, Patrick Lauer wrote: On Sunday 14 September 2014 15:42:15 hasufell wrote: Patrick Lauer: Are we going to disallow merge commits and ask devs to rebase local changes in order to keep the history clean? Is that going to be sane with our commit frequency? You have to merge or rebase anyway in case of a push conflict, so the only difference is the method and the effect on the history. Currently... CVS allows you to run repoman on an outdated tree and push broken ebuilds with repoman being happy. Git will not allow this. iow, git doesn't allow people to work on more than one item at a time? It does. I think we really have to write up a step-by-step guide (not just workflow policies) for people who have never seriously worked with git. On the other hand... there are thousands of tutorials on the net already. For the workflow model, I already have created a draft which is in no way finished or even correct and there are still some controversially discussed issues. https://wiki.gentoo.org/wiki/Gentoo_git_workflow As a prospective Gentoo developer, having a guide around meant specifically for Gentoo's practices would be incredibly helpful. I use git in my own hobby development and learned from Pro Git, et al, but it'd still be really nice to have, and give developers a place to point to if a new developer is having troubles. Just my 2¢. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: My masterplan for git migration (+ looking for infra to test it)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/16/2014 05:18 AM, hasufell wrote: Ulrich Mueller: ChangeLogs are aimed at users Did any1 ask them if they care? If the tree switches to git and there's an option within Portage/emerge to fetch via git instead of rsync, then I'd rather rely on `git log` than a bunch of scattered files. -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJUGRVnAAoJEJUrb08JgYgH1pQH/RiBmtnUvewDY+dm1cdbAdWb A8YXcHDHhYnVtll3x7hB+YphKLNYBN+baLLiKXHAR4LaWIfc+Z0NHMpfN3pNQTwZ o3XjzShWMhZ9Z5mTafPuFgR1f+sAuqSG0lOhMm3tHwKmBEHt3fh2bnAZVkGtnJRE L/xDCU5sniGPJCLhXBaPfU3om99xeEQtahXWR+rVHj64h93t9Cb1hHIlWRvjPzDT M5kC9Rz/BS1wO4mwPqi/jW5mbQnLUhcy7y4OSszQeAMyroCIhkxwwKLeWES62XQr bo6AKqv1SKMFVYIgYVRei0iTXbQ2/pWzlpatM11G6djqMtTvDlMR7f3wPbAiw2U= =EIKj -END PGP SIGNATURE-
Re: [gentoo-portage-dev] [PATCH] per package environment: generalize the mechanism to be profile specific
I moved the profile attributes detection logic into the python side as
suggested (much cleaner).
The updated patch is attached.
Thanks,
Bertrand
On Mon, Sep 15, 2014 at 1:40 PM, Zac Medico zmed...@gentoo.org wrote:
On 09/15/2014 11:42 AM, Bertrand Simonnet wrote:
I replaced the FEATURES gate by a profile-formats option.
I added some logic to find layout.conf for a given profile path. (find
the first
parent dir named profiles then check ../metadata/layout.conf)
Is there a corner case that I missed?
I all looks good to me, except that I don't like the way that
__profile_env_enabled parses the layout.conf for each directory, since
it's terribly inefficient. It would be much better to parse the
layout.conf data on the python side, and pass it to bash as an
environment variable. For example, the PORTAGE_ECLASS_LOCATIONS variable
is generated in python such that eval can be used to convert it to an
array. We can add a similar array variable called
PORTAGE_PROFILE_ATTRIBUTES, and each element of the array will
correspond to the profile path at the same index in the path_array
variable.
--
Thanks,
Zac
From 921290494664b356e7bb82f5a5a86b40287a49d2 Mon Sep 17 00:00:00 2001
From: Bertrand SIMONNET bsimon...@chromium.org
Date: Fri, 12 Sep 2014 05:07:20 -0700
Subject: [PATCH] Make env/ bash scripts profile specific
This generalize the /etc/portage/env mechanism to be stackable with profiles.
ebuild.sh will walk the profiles and sources scripts in env/ following the same
matching rules as for /etc/portage/env.
Change-Id: I16bcd75790213d2204f83d4aa6e8b910f8829b6e
---
bin/ebuild.sh | 86 ++
bin/save-ebuild-env.sh | 1 +
.../package/ebuild/_config/special_env_vars.py | 2 +-
pym/portage/package/ebuild/config.py | 20 +
pym/portage/package/ebuild/doebuild.py | 1 +
pym/portage/repository/config.py | 2 +-
6 files changed, 78 insertions(+), 34 deletions(-)
diff --git a/bin/ebuild.sh b/bin/ebuild.sh
index be044e0..26b9190 100755
--- a/bin/ebuild.sh
+++ b/bin/ebuild.sh
@@ -67,11 +67,11 @@ unset BASH_ENV
# earlier portage versions do not detect a version change in this case
# ( to ) and therefore they try execute an incompatible version of
# ebuild.sh during the upgrade.
-export PORTAGE_BZIP2_COMMAND=${PORTAGE_BZIP2_COMMAND:-bzip2}
+export PORTAGE_BZIP2_COMMAND=${PORTAGE_BZIP2_COMMAND:-bzip2}
# These two functions wrap sourcing and calling respectively. At present they
# perform a qa check to make sure eclasses and ebuilds and profiles don't mess
-# with shell opts (shopts). Ebuilds/eclasses changing shopts should reset them
+# with shell opts (shopts). Ebuilds/eclasses changing shopts should reset them
# when they are done.
__qa_source() {
@@ -275,7 +275,7 @@ inherit() {
set +f
__qa_source $location || die died sourcing $location in inherit()
-
+
#turn off glob expansion
set -f
@@ -290,7 +290,7 @@ inherit() {
[ ${B_IUSE+set} = set ] IUSE=${B_IUSE}
[ ${B_IUSE+set} = set ] || unset IUSE
-
+
[ ${B_REQUIRED_USE+set} = set ] REQUIRED_USE=${B_REQUIRED_USE}
[ ${B_REQUIRED_USE+set} = set ] || unset REQUIRED_USE
@@ -369,45 +369,67 @@ __source_all_bashrcs() {
save_IFS
IFS=$'\n'
local path_array=($PROFILE_PATHS)
+ local profile_attributes=($PORTAGE_PROFILE_ATTRIBUTES)
+ local profile_path
restore_IFS
- for x in ${path_array[@]} ; do
- [ -f $x/profile.bashrc ] __qa_source $x/profile.bashrc
+ for i in ${!path_array[@]} ; do
+ profile_path=${path_array[$i]}
+ [ -f ${profile_path}/profile.bashrc ] \
+__qa_source ${profile_path}/profile.bashrc
+ [[ ${profile_attributes[$i]} == *profile-env* ]] \
+__source_env_files ${profile_path}/env
done
fi
- if [ -r ${PORTAGE_BASHRC} ] ; then
- if [ $PORTAGE_DEBUG != 1 ] || [ ${-/x/} != $- ]; then
- source ${PORTAGE_BASHRC}
- else
- set -x
- source ${PORTAGE_BASHRC}
- set +x
- fi
- fi
+ # The user's bashrc is the ONLY non-portage bit of code
+ # that can change shopts without a QA violation.
+ __try_source ${PORTAGE_BASHRC} no_qa
if [[ $EBUILD_PHASE != depend ]] ; then
- # The user's bashrc is the ONLY non-portage bit of code that can
- # change shopts without a QA violation.
- for x in ${PM_EBUILD_HOOK_DIR}/${CATEGORY}/{${PN},${PN}:${SLOT%/*},${P},${PF}}; do
- if [ -r ${x} ]; then
-# If $- contains x, then tracing has already been enabled
-# elsewhere for some reason. We preserve it's state so as
-# not to interfere.
-if [ $PORTAGE_DEBUG != 1 ] || [ ${-/x/} != $- ]; then
- source ${x}
-else
- set -x
- source ${x}
- set +x
-fi
- fi
- done
+ __source_env_files ${PM_EBUILD_HOOK_DIR} no_qa
fi
[ ! -z ${OCC} ] export CC=${OCC}
[ ! -z ${OCXX} ] export CXX=${OCXX}
}
+# @FUNCTION: __source_env_files
+# @DESCRIPTION:
+# Source the files relevant to
[gentoo-portage-dev] [PATCH] _solve_non_slot_operator_slot_conflicts: fix bug #510270
This fixes an IndexError in _solve_non_slot_operator_slot_conflicts which occurs when none of the conflict packages matched a particular atom. This typically means that autounmask broke a USE-dep, but it could also be due to the slot not matching due to multislot (bug #220341). Either way, don't try to solve this conflict. Instead, force all of the associated conflict nodes into the graph so that they are protected from removal by the conflict solver. X-Gentoo-Bug: 510270 X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=510270 --- pym/_emerge/depgraph.py| 14 + pym/portage/tests/resolver/ResolverPlayground.py | 9 .../tests/resolver/test_autounmask_use_breakage.py | 63 ++ 3 files changed, 86 insertions(+) create mode 100644 pym/portage/tests/resolver/test_autounmask_use_breakage.py diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py index cc87d9f..b31f90c 100644 --- a/pym/_emerge/depgraph.py +++ b/pym/_emerge/depgraph.py @@ -1059,6 +1059,7 @@ class depgraph(object): def __str__(self): return (%s) % ,.join(str(pkg) for pkg in self) + non_matching_forced = set() for conflict in conflicts: if debug: writemsg_level( conflict:\n, level=logging.DEBUG, noiselevel=-1) @@ -1105,6 +1106,18 @@ class depgraph(object): continue elif len(matched) == 1: conflict_graph.add(matched[0], parent) + elif len(matched) == 0: + # This typically means that autounmask broke a + # USE-dep, but it could also be due to the slot + # not matching due to multislot (bug #220341). + # Either way, don't try to solve this conflict. + # Instead, force them all into the graph so that + # they are protected from removal. + non_matching_forced.update(conflict) + if debug: + for pkg in conflict: + writemsg_level( non-match: %s\n % pkg, + level=logging.DEBUG, noiselevel=-1) else: # More than one packages matched, but not all. conflict_graph.add(or_tuple(matched), parent) @@ -1125,6 +1138,7 @@ class depgraph(object): # Now select required packages. Collect them in the # 'forced' set. forced = set([non_conflict_node]) + forced.update(non_matching_forced) unexplored = set([non_conflict_node]) # or_tuples get special handling. We first explore # all packages in the hope of having forced one of diff --git a/pym/portage/tests/resolver/ResolverPlayground.py b/pym/portage/tests/resolver/ResolverPlayground.py index 77a5b5c..b1974d7 100644 --- a/pym/portage/tests/resolver/ResolverPlayground.py +++ b/pym/portage/tests/resolver/ResolverPlayground.py @@ -549,6 +549,7 @@ class ResolverPlaygroundTestCase(object): self.all_permutations = kwargs.pop(all_permutations, False) self.ignore_mergelist_order = kwargs.pop(ignore_mergelist_order, False) self.ambiguous_merge_order = kwargs.pop(ambiguous_merge_order, False) + self.ambiguous_slot_collision_solutions = kwargs.pop(ambiguous_slot_collision_solutions, False) self.check_repo_names = kwargs.pop(check_repo_names, False) self.merge_order_assertions = kwargs.pop(merge_order_assertions, False) @@ -664,6 +665,14 @@ class ResolverPlaygroundTestCase(object): str((node1, node2))) + \ , got: + str(got)) + elif key == slot_collision_solutions and \ + self.ambiguous_slot_collision_solutions: + # Tests that use all_permutations can have multiple + # outcomes here. + for x in expected: + if x == got: + expected = x + break elif key in (unstable_keywords, needed_p_mask_changes, unsatisfied_deps) and
Re: [gentoo-portage-dev] [PATCH] per package environment: generalize the mechanism to be profile specific
On 09/16/2014 11:17 AM, Bertrand Simonnet wrote: I moved the profile attributes detection logic into the python side as suggested (much cleaner). Thanks, that's better. I've got a couple more issues though: 1) Like global functions, global variables should also be unset in __save_ebuild_env. So, we should unset PORTAGE_PROFILE_ATTRIBUTES there. 2) Instead of having _get_profile_attributes read the layout.conf files directly, it would be nicer if we could integrated it with the LocationsManager layout.conf parsing. For example, see the intersecting_repos code inside the _addProfile method. -- Thanks, Zac
Re: [gentoo-portage-dev] [PATCH] per package environment: generalize the mechanism to be profile specific
On Tue, Sep 16, 2014 at 12:17 PM, Zac Medico zmed...@gentoo.org wrote:
On 09/16/2014 11:17 AM, Bertrand Simonnet wrote:
I moved the profile attributes detection logic into the python side as
suggested (much cleaner).
Thanks, that's better. I've got a couple more issues though:
1) Like global functions, global variables should also be unset in
__save_ebuild_env. So, we should unset PORTAGE_PROFILE_ATTRIBUTES there.
Done
2) Instead of having _get_profile_attributes read the layout.conf files
directly, it would be nicer if we could integrated it with the
LocationsManager layout.conf parsing. For example, see the
intersecting_repos code inside the _addProfile method.
This is much better. :)
--
Thanks,
Zac
From 4bf1ee98bb97136e3f6f2182e205aed7ca597640 Mon Sep 17 00:00:00 2001
From: Bertrand SIMONNET bsimon...@chromium.org
Date: Fri, 12 Sep 2014 05:07:20 -0700
Subject: [PATCH] Make env/ bash scripts profile specific
This generalize the /etc/portage/env mechanism to be stackable with profiles.
ebuild.sh will walk the profiles and sources scripts in env/ following the same
matching rules as for /etc/portage/env.
Change-Id: I16bcd75790213d2204f83d4aa6e8b910f8829b6e
---
bin/ebuild.sh | 86 ++
bin/save-ebuild-env.sh | 3 +-
.../package/ebuild/_config/LocationsManager.py | 6 ++
.../package/ebuild/_config/special_env_vars.py | 2 +-
pym/portage/package/ebuild/config.py | 2 +
pym/portage/package/ebuild/doebuild.py | 2 +
pym/portage/repository/config.py | 2 +-
7 files changed, 68 insertions(+), 35 deletions(-)
diff --git a/bin/ebuild.sh b/bin/ebuild.sh
index be044e0..26b9190 100755
--- a/bin/ebuild.sh
+++ b/bin/ebuild.sh
@@ -67,11 +67,11 @@ unset BASH_ENV
# earlier portage versions do not detect a version change in this case
# ( to ) and therefore they try execute an incompatible version of
# ebuild.sh during the upgrade.
-export PORTAGE_BZIP2_COMMAND=${PORTAGE_BZIP2_COMMAND:-bzip2}
+export PORTAGE_BZIP2_COMMAND=${PORTAGE_BZIP2_COMMAND:-bzip2}
# These two functions wrap sourcing and calling respectively. At present they
# perform a qa check to make sure eclasses and ebuilds and profiles don't mess
-# with shell opts (shopts). Ebuilds/eclasses changing shopts should reset them
+# with shell opts (shopts). Ebuilds/eclasses changing shopts should reset them
# when they are done.
__qa_source() {
@@ -275,7 +275,7 @@ inherit() {
set +f
__qa_source $location || die died sourcing $location in inherit()
-
+
#turn off glob expansion
set -f
@@ -290,7 +290,7 @@ inherit() {
[ ${B_IUSE+set} = set ] IUSE=${B_IUSE}
[ ${B_IUSE+set} = set ] || unset IUSE
-
+
[ ${B_REQUIRED_USE+set} = set ] REQUIRED_USE=${B_REQUIRED_USE}
[ ${B_REQUIRED_USE+set} = set ] || unset REQUIRED_USE
@@ -369,45 +369,67 @@ __source_all_bashrcs() {
save_IFS
IFS=$'\n'
local path_array=($PROFILE_PATHS)
+ local profile_attributes=($PORTAGE_PROFILE_ATTRIBUTES)
+ local profile_path
restore_IFS
- for x in ${path_array[@]} ; do
- [ -f $x/profile.bashrc ] __qa_source $x/profile.bashrc
+ for i in ${!path_array[@]} ; do
+ profile_path=${path_array[$i]}
+ [ -f ${profile_path}/profile.bashrc ] \
+__qa_source ${profile_path}/profile.bashrc
+ [[ ${profile_attributes[$i]} == *profile-env* ]] \
+__source_env_files ${profile_path}/env
done
fi
- if [ -r ${PORTAGE_BASHRC} ] ; then
- if [ $PORTAGE_DEBUG != 1 ] || [ ${-/x/} != $- ]; then
- source ${PORTAGE_BASHRC}
- else
- set -x
- source ${PORTAGE_BASHRC}
- set +x
- fi
- fi
+ # The user's bashrc is the ONLY non-portage bit of code
+ # that can change shopts without a QA violation.
+ __try_source ${PORTAGE_BASHRC} no_qa
if [[ $EBUILD_PHASE != depend ]] ; then
- # The user's bashrc is the ONLY non-portage bit of code that can
- # change shopts without a QA violation.
- for x in ${PM_EBUILD_HOOK_DIR}/${CATEGORY}/{${PN},${PN}:${SLOT%/*},${P},${PF}}; do
- if [ -r ${x} ]; then
-# If $- contains x, then tracing has already been enabled
-# elsewhere for some reason. We preserve it's state so as
-# not to interfere.
-if [ $PORTAGE_DEBUG != 1 ] || [ ${-/x/} != $- ]; then
- source ${x}
-else
- set -x
- source ${x}
- set +x
-fi
- fi
- done
+ __source_env_files ${PM_EBUILD_HOOK_DIR} no_qa
fi
[ ! -z ${OCC} ] export CC=${OCC}
[ ! -z ${OCXX} ] export CXX=${OCXX}
}
+# @FUNCTION: __source_env_files
+# @DESCRIPTION:
+# Source the files relevant to the current package from the given path.
+# If $2 is not 'no_qa', all the scripts are sourced with __qa_source.
+__source_env_files() {
+ for x in ${1}/${CATEGORY}/{${PN},${PN}:${SLOT%/*},${P},${PF}}; do
+ __try_source ${x} ${2}
+ done
+}
+
+# @FUNCTION: __try_source
+# @DESCRIPTION:
+# If the path given as argument exists, source the file while
Re: [gentoo-portage-dev] [PATCH] per package environment: generalize the mechanism to be profile specific
On 09/16/2014 02:37 PM, Bertrand Simonnet wrote: On Tue, Sep 16, 2014 at 12:17 PM, Zac Medico zmed...@gentoo.org mailto:zmed...@gentoo.org wrote: On 09/16/2014 11:17 AM, Bertrand Simonnet wrote: I moved the profile attributes detection logic into the python side as suggested (much cleaner). Thanks, that's better. I've got a couple more issues though: 1) Like global functions, global variables should also be unset in __save_ebuild_env. So, we should unset PORTAGE_PROFILE_ATTRIBUTES there. Done 2) Instead of having _get_profile_attributes read the layout.conf files directly, it would be nicer if we could integrated it with the LocationsManager layout.conf parsing. For example, see the intersecting_repos code inside the _addProfile method. This is much better. :) I all looks very well done to me now. I'd encourage others on the list to review it now, in case there's anything that I missed. -- Thanks, Zac
