Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-23 Thread Ulrich Mueller
> On Mon, 22 Sep 2014, W Trevor King wrote:

> There's no Signed-off-by on the commits adding the DCO to the Linux
> tree ;).  The only information I can find claiming copyright and
> licensing by one of the DCO authors is at
> http://developercertificate.org/.  I suppose you could alter the DCO
> and claim it's under a different license, but the Linux Foundation
> lawers wrote the thing, so I think it's more respectful to take them
> at their word or just write your own certificate from scratch.

The Linux Foundation was founded only in 2007, so it is not possible
that its lawyers wrote the DCO, which (version 1.1) is from 2005.

In fact, the OSDL, as one of the predecesors of the Linux Foundation,
had released the DCO under these terms [1]:

   © 2005 Open Source Development Labs, Inc. The Developer's
   Certificate of Origin 1.1 is licensed under a Creative Commons
   Attribution-ShareAlike 2.5 License. If you modify you must use a
   name or title distinguishable from "Developer's Certificate of
   Origin" or "DCO" or any confusingly similar name.

Certainly the copyright holders can re-release it under more
restrictive terms, but they cannot retroactively take away from us the
rights that they have granted us under CC BY-SA 2.5.

Ulrich


[1] 
https://web.archive.org/web/20060524185355/http://www.osdlab.org/newsroom/press_releases/2004/2004_05_24_dco.html


pgpsLAnMLVkd_.pgp
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread W. Trevor King
On Mon, Sep 22, 2014 at 04:56:58PM -0400, Rich Freeman wrote:
> In any case, I don't think it is necessary to actually modify the DCO.

Ah, good.  Then the verbatim copy license is sufficient, and we don't
need to decide if the GPLv2 with Linus' exception applies.

> I don't believe that it requires redistributing commit messages.

I don't either, it just means that you *can* sign the DCO for a commit
that you got from someone who's also signed the DCO for that commit.
We just went down this licensing track because of:

On Mon, Sep 22, 2014 at 02:13:53PM -0400, Rich Freeman wrote:
> Perhaps the c clause should be clarified that the source files
> themselves were not modified - not the commit message.

I thought that sounded like you were suggesting a modification to the
DCO, so I pointed out that I don't think that's legal.  If you were
just suggesting some “we interpret clause (c) to mean …” comment on
the wiki or somewhere else, that clearly is legal, but I don't see the
point.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Rich Freeman
On Mon, Sep 22, 2014 at 3:43 PM, W. Trevor King  wrote:
> There's no Signed-off-by on the commits adding the DCO to the Linux
> tree ;).  The only information I can find claiming copyright and
> licensing by one of the DCO authors is at
> http://developercertificate.org/.  I suppose you could alter the DCO
> and claim it's under a different license, but the Linux Foundation
> lawers wrote the thing, so I think it's more respectful to take them
> at their word or just write your own certificate from scratch.

The license is declared in /usr/src/linux/COPYING.

Or, are you suggesting that it is illegal to redistribute the kernel
tarball?  The tarball is distributed under the GPL unless otherwise
stated, and nothing in that file containing the DCO states otherwise.
Certainly the tarball contains no license that only allows unmodified
redistribution.

The fact that the same text is published multiple times isn't an
issue.  You can publish code under as many incompatible licenses as
you wish.  Recipients have to follow the license they received it
under, and if they received it under more than one then they basically
get to choose.  Licenses GIVE rights, they don't take them away.
Absent a license you wouldn't be able to redistribute a file
unmodified or otherwise.

In any case, I don't think it is necessary to actually modify the DCO.
I don't believe that it requires redistributing commit messages.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread W. Trevor King
On Mon, Sep 22, 2014 at 03:35:29PM -0400, Rich Freeman wrote:
> On Mon, Sep 22, 2014 at 3:27 PM, W. Trevor King wrote:
> > On Mon, Sep 22, 2014 at 03:13:35PM -0400, Rich Freeman wrote:
> >> On Mon, Sep 22, 2014 at 2:28 PM, W. Trevor King wrote:
> >> > On Mon, Sep 22, 2014 at 02:13:53PM -0400, Rich Freeman wrote:
> >> >> Perhaps the c clause should be clarified that the source files
> >> >> themselves were not modified - not the commit message.
> >> >
> >> > The DCO text is verbatim copies only [1], so I don't think
> >> > adjusting clauses is legal.
> >>
> >> I copied it from /usr/src/linux/Documentation/SubmittingPatches
> >> which is GPLv2, as far as I can tell.
> >
> > Luis R. Rodriguez and I spent some time trying to track this down with
> > the authors while I was factoring the signed-off-by documentation out
> > into a stand-alone repository [1,2].  There was some debate about
> > whether the text was copyrightable, but the explicit copyright claim
> > and license on the Linux Foundation's DCO page [3] settles it for me.
> 
> Great to hear that it settles it for you, but as far as I can tell,
> the Linux Foundation has released it under the GPL and continues to do
> so to this day.  I suppose they can sue me if they don't agree, not
> that I can see why they would want to.  :)

There's no Signed-off-by on the commits adding the DCO to the Linux
tree ;).  The only information I can find claiming copyright and
licensing by one of the DCO authors is at
http://developercertificate.org/.  I suppose you could alter the DCO
and claim it's under a different license, but the Linux Foundation
lawers wrote the thing, so I think it's more respectful to take them
at their word or just write your own certificate from scratch.

> >> > Personally, I don't think the maintainer appending their s-o-b to
> >> > the user's commit is all that important (certainly not worth
> >> > blowing away the user's signature) when they can just sign and
> >> > s-o-b an explicit merge commit.
> >>
> >> Agree.  No need to modify the original commit.
> >
> > So the policy in the wiki should be:
> >
> >   “Don't clobber the user's signature on a commit, even to add your
> >   Signed-off-by.  Instead, explicitly merge signed user commits, or
> >   have the user reroll the commit with your tweaks and re-sign it.”
> 
> I disagree with this.
> 
> I have no objections to keeping the original commit.  However, I do
> object to requiring that the original commit being preserved.

So, “You don't need to clobber….  Instead, you can explicitly….”  Then
it's clear that clobbering user sigs is allowed, even if it's not very
nice ;).

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Rich Freeman
On Mon, Sep 22, 2014 at 3:27 PM, W. Trevor King  wrote:
> On Mon, Sep 22, 2014 at 03:13:35PM -0400, Rich Freeman wrote:
>> On Mon, Sep 22, 2014 at 2:28 PM, W. Trevor King  wrote:
>> > On Mon, Sep 22, 2014 at 02:13:53PM -0400, Rich Freeman wrote:
>> >> Perhaps the c clause should be clarified that the source files
>> >> themselves were not modified - not the commit message.
>> >
>> > The DCO text is verbatim copies only [1], so I don't think
>> > adjusting clauses is legal.
>>
>> I copied it from /usr/src/linux/Documentation/SubmittingPatches
>> which is GPLv2, as far as I can tell.
>
> Luis R. Rodriguez and I spent some time trying to track this down with
> the authors while I was factoring the signed-off-by documentation out
> into a stand-alone repository [1,2].  There was some debate about
> whether the text was copyrightable, but the explicit copyright claim
> and license on the Linux Foundation's DCO page [3] settles it for me.

Great to hear that it settles it for you, but as far as I can tell,
the Linux Foundation has released it under the GPL and continues to do
so to this day.  I suppose they can sue me if they don't agree, not
that I can see why they would want to.  :)

>
>> > Personally, I don't think the maintainer appending their s-o-b to
>> > the user's commit is all that important (certainly not worth
>> > blowing away the user's signature) when they can just sign and
>> > s-o-b an explicit merge commit.
>>
>> Agree.  No need to modify the original commit.
>
> So the policy in the wiki should be:
>
>   “Don't clobber the user's signature on a commit, even to add your
>   Signed-off-by.  Instead, explicitly merge signed user commits, or
>   have the user reroll the commit with your tweaks and re-sign it.”

I disagree with this.

I have no objections to keeping the original commit.  However, I do
object to requiring that the original commit being preserved.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread W. Trevor King
On Mon, Sep 22, 2014 at 03:13:35PM -0400, Rich Freeman wrote:
> On Mon, Sep 22, 2014 at 2:28 PM, W. Trevor King  wrote:
> > On Mon, Sep 22, 2014 at 02:13:53PM -0400, Rich Freeman wrote:
> >> Perhaps the c clause should be clarified that the source files
> >> themselves were not modified - not the commit message.
> >
> > The DCO text is verbatim copies only [1], so I don't think
> > adjusting clauses is legal.
> 
> I copied it from /usr/src/linux/Documentation/SubmittingPatches
> which is GPLv2, as far as I can tell.

Luis R. Rodriguez and I spent some time trying to track this down with
the authors while I was factoring the signed-off-by documentation out
into a stand-alone repository [1,2].  There was some debate about
whether the text was copyrightable, but the explicit copyright claim
and license on the Linux Foundation's DCO page [3] settles it for me.

> > Personally, I don't think the maintainer appending their s-o-b to
> > the user's commit is all that important (certainly not worth
> > blowing away the user's signature) when they can just sign and
> > s-o-b an explicit merge commit.
> 
> Agree.  No need to modify the original commit.

So the policy in the wiki should be:

  “Don't clobber the user's signature on a commit, even to add your
  Signed-off-by.  Instead, explicitly merge signed user commits, or
  have the user reroll the commit with your tweaks and re-sign it.”

Cheers,
Trevor

[1]: https://github.com/wking/signed-off-by
[2]: http://www.do-not-panic.com/2014/02/developer-certificate-of-origin.html
[3]: http://developercertificate.org/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Rich Freeman
On Mon, Sep 22, 2014 at 2:28 PM, W. Trevor King  wrote:
> On Mon, Sep 22, 2014 at 02:13:53PM -0400, Rich Freeman wrote:
>> Perhaps the c clause should be clarified that the source files
>> themselves were not modified - not the commit message.
>
> The DCO text is verbatim copies only [1], so I don't think adjusting
> clauses is legal.

I copied it from /usr/src/linux/Documentation/SubmittingPatches which
is GPLv2, as far as I can tell.

But, I don't think the text really applies to the commits - just the
code itself.  But, whatever.

> And if you're modifying neither the source files
> nor the commit message, I'm not sure where you're suggesting the
> Signed-off-by go.  Or are you saying that when a maintainer adds their
> s-o-b and blows away the user's signature, they should just say “don't
> worry, this is still pretty much what the user signed”?

Yes.  The user's commit will probably not end up in the tree most of
the time.  Not that I object to them being there.

>  Personally, I
> don't think the maintainer appending their s-o-b to the user's commit
> is all that important (certainly not worth blowing away the user's
> signature) when they can just sign and s-o-b an explicit merge commit.

Agree.  No need to modify the original commit.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread W. Trevor King
On Mon, Sep 22, 2014 at 02:13:53PM -0400, Rich Freeman wrote:
> Perhaps the c clause should be clarified that the source files
> themselves were not modified - not the commit message.

The DCO text is verbatim copies only [1], so I don't think adjusting
clauses is legal.  And if you're modifying neither the source files
nor the commit message, I'm not sure where you're suggesting the
Signed-off-by go.  Or are you saying that when a maintainer adds their
s-o-b and blows away the user's signature, they should just say “don't
worry, this is still pretty much what the user signed”?  Personally, I
don't think the maintainer appending their s-o-b to the user's commit
is all that important (certainly not worth blowing away the user's
signature) when they can just sign and s-o-b an explicit merge commit.

> I don't have a problem with preserving contributor commits via merge
> commits, but I don't think that is the general proposed workflow.

The wiki has [2]:

  don't do complicated rebases to avoid a merge commit at all cost (it
  may even cause losing information, e.g. user signatures)

which makes sense to me.

Cheers,
Trevor

[1]: http://developercertificate.org/
[2]: 
https://wiki.gentoo.org/index.php?title=Gentoo_git_workflow&redirect=no#atomicity

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Rich Freeman
On Mon, Sep 22, 2014 at 12:52 PM, W. Trevor King  wrote:
> On Mon, Sep 22, 2014 at 11:29:52AM -0400, Rich Freeman wrote:
>> On Mon, Sep 22, 2014 at 10:50 AM, Ulrich Mueller wrote:
>> > Another issue, should we require "Signed-off-by:" lines? At least
>> > for things that are contributed by users?
>> >
>> > …
>>
>> Thanks for bringing this up.  I had circulated the start of a
>> proposal on this a year ago:
>> http://dev.gentoo.org/~rich0/copyrightpolicy.xml
>
> The (c) clause (“I got this patch from someone else who'd signed the
> DCO for it”) leads to chains like:
>
>   Signed-off-by: A. U. Thor 
>   Signed-off-by: Some Maintainer 
>   …
>
> as the patch percolates up to the main repository.  In Gentoo, that's
> probably going to be just a Gentoo dev, or an external contributor
> plus a Gentoo dev.  The multiple-signoffs version is not going to play
> well with signed commits, because if A. U. Thor signed his commit
> (with just his Signed-off-by), Some Maintainer will not be able to add
> her Signed-off-by without dropping Thor's commit signature.  My
> suggested solution here is to take the same approach we're suggesting
> for commit signatures, and just have the maintainer add their
> Signed-off-by to an explicit merge commit pulling in the contributor's
> work.

Perhaps the c clause should be clarified that the source files
themselves were not modified - not the commit message.

I don't have a problem with preserving contributor commits via merge
commits, but I don't think that is the general proposed workflow.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread W. Trevor King
On Mon, Sep 22, 2014 at 11:29:52AM -0400, Rich Freeman wrote:
> On Mon, Sep 22, 2014 at 10:50 AM, Ulrich Mueller wrote:
> > Another issue, should we require "Signed-off-by:" lines? At least
> > for things that are contributed by users?
> >
> > …
> 
> Thanks for bringing this up.  I had circulated the start of a
> proposal on this a year ago:
> http://dev.gentoo.org/~rich0/copyrightpolicy.xml

The (c) clause (“I got this patch from someone else who'd signed the
DCO for it”) leads to chains like:

  Signed-off-by: A. U. Thor 
  Signed-off-by: Some Maintainer 
  …

as the patch percolates up to the main repository.  In Gentoo, that's
probably going to be just a Gentoo dev, or an external contributor
plus a Gentoo dev.  The multiple-signoffs version is not going to play
well with signed commits, because if A. U. Thor signed his commit
(with just his Signed-off-by), Some Maintainer will not be able to add
her Signed-off-by without dropping Thor's commit signature.  My
suggested solution here is to take the same approach we're suggesting
for commit signatures, and just have the maintainer add their
Signed-off-by to an explicit merge commit pulling in the contributor's
work.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Rich Freeman
On Mon, Sep 22, 2014 at 10:50 AM, Ulrich Mueller  wrote:
>> On Mon, 22 Sep 2014, hasufell  wrote:
>
 https://wiki.gentoo.org/wiki/Gentoo_git_workflow
>
 But so far, not many people have been particularly interested in
 the details of these things. I'm also not sure if the ML is the
 right way to figure out these details.
>
> Another issue, should we require "Signed-off-by:" lines? At least for
> things that are contributed by users?
>
> We could just use the same rules for it as outlined in Linux
> documentation (see Documentation/SubmittingPatches, especially
> "Developer's Certificate of Origin 1.1").
>

Thanks for bringing this up.  I had circulated the start of a proposal
on this a year ago:
http://dev.gentoo.org/~rich0/copyrightpolicy.xml

We didn't really take it further, but instituting just the DCO
component of this policy using git signed-off-by lines should be
fairly non-controversial.  The FLA probably needs some review/etc but
the intent was for that to be optional.  The DCO is based on the linux
DCO.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Ulrich Mueller
> On Mon, 22 Sep 2014, hasufell  wrote:

>>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow

>>> But so far, not many people have been particularly interested in
>>> the details of these things. I'm also not sure if the ML is the
>>> right way to figure out these details.

Another issue, should we require "Signed-off-by:" lines? At least for
things that are contributed by users?

We could just use the same rules for it as outlined in Linux
documentation (see Documentation/SubmittingPatches, especially
"Developer's Certificate of Origin 1.1").

Ulrich


pgp6utUPT5W0t.pgp
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread hasufell
On 09/22/2014 08:40 AM, Ulrich Mueller wrote:
>> On Mon, 22 Sep 2014, hasufell  wrote:
> 
>> Ulrich Mueller:
>>> | • atomic commits (one logical change)
>>>
>>> A version bump plus cleaning up older ebuilds will be considered
>>> one logical change, I suppose?
> 
>> I'd consider it two logical changes (e.g. imagine a user complaining
>> about ebuild removal... you cannot easily revert it if it's not a
>> separate commit). But I don't have a strong opinion on that and I'm
>> not sure if we can enforce commit rules in such fine-grained
>> details, can we?
> 
>> Do you think this should be added explicitly?
> 
> It is a very common example that should be mentioned.
> 

Another example that just crossed my mind is ebuild bumps that also
modify profiles/ (e.g. package.mask, because a dependency is also masked).

I think I was initially assuming that it would be one commit since both
things are related to the bump, but it also has advantages to split that up.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Tom Wijsman
On Mon, 22 Sep 2014 08:56:04 +0200
Michał Górny  wrote:

> How can we distinguish between accidental and intentional stable
> keyword removals? :)

(The lack of) Proper commit messages that point out those removals! ;)

But well, yeah, that'll require consistency and so on...



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-22 Thread Tom Wijsman
On Fri, 19 Sep 2014 16:03:57 +0200
Tobias Klausmann  wrote:

> As I pointed out, getting the right code into the tree is not the
> problem here. It is extra work over the current way of doing it
> (since I need to deal with a local commit that can't be ff'd or
> rebased as git is very line-oriented.

It can be rebased when you use branches, which is less line-oriented;
yeah, that indeed is extra work over plain single commit. This can be
improved with shell aliases or functions, to spare out on key presses...

> On top of the extra work, there have been several mentions that
> only meaning ful merge-commits are wanted in the tree (or not
> wanted at all). AIUI, avoiding them in the keywording/stabilizing
> phase is currently very difficult, unless we split KEYWORDS into
> separate lines or find another mechanism (like having the
> maintainer/keyword-requestor do all the edits after the archs
> sign off on them).

Not a problem when you rebase.
 
> I'd be delighted to hear a simpler solution that only involves
> doing the semantic merge (like we do now with CVS).

Create one or two shell aliases or functions for pulling that do:

 1) stashes your local modifications
 2) create a new branch with your local commits
 3) reset the master branch to the last upstream commit
 4) pulls new changes into the master branch
 5) rebases your new branch onto master
 6) remove the master branch and rename your new branch to it
 7) applies the stash from step (1)

Given that the rebase is interactive, 1-5 and 5-7 will probably be
separate shell aliases or functions; nonetheless, this takes away a ton
of manual typing giving you more time for where the actual work is.

I would advise against this automation for other projects; but given the
plain single commit nature we're used to, this works for Portage tree.

> And at least in my case, collisions during keywording are fairly
> common, and will be even more so with git since fetch/pull are
> slower than for a CVS subdir (since git checks the whole tree for
> local changes, not just the current subtree, AIUI).

Has this been experimented with? Is the Portage tree that big? Afaik
Git does a lot to be fast, whereas I experience CVS to be slow; I would
be surprised if Git were to be slower than faster compared to CVS.

> Again, correct me if I'm wrong. I've been using git for ~4 years,
> but only in single-developer mode. And even there I managed to
> make a mess of repo histories :-/ Fortunately, nobody cares about
> my stuff.

Reading into Git workflows and documentation about stashing,
rebasing, ... can make a world of difference to turn mess into a gem.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Michał Górny
Dnia 2014-09-21, o godz. 23:36:58
Peter Stuge  napisał(a):

> Jonathan Callen wrote:
> > the correct response would be to ensure that the final commit
> > pushed (whether it be a merge commit or rebased) contains the
> > stabilization for both arches
> 
> I think this is one of the things to check in a post-receive or
> post-update hook. What is the easiest way to access keywords out of
> an ebuild - which does *not* require sourcing the ebuild in a shell?

How can we distinguish between accidental and intentional stable
keyword removals? :) However, this could probably go as part of atomic
change policy -- you can't stabilize and de-stabilize in one commit.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Ulrich Mueller
> On Mon, 22 Sep 2014, hasufell  wrote:

> Ulrich Mueller:
>> | • atomic commits (one logical change)
>> 
>> A version bump plus cleaning up older ebuilds will be considered
>> one logical change, I suppose?

> I'd consider it two logical changes (e.g. imagine a user complaining
> about ebuild removal... you cannot easily revert it if it's not a
> separate commit). But I don't have a strong opinion on that and I'm
> not sure if we can enforce commit rules in such fine-grained
> details, can we?

> Do you think this should be added explicitly?

It is a very common example that should be mentioned.

>> | • for commits that affect only a single package, prepend
>> |   "CATEGORY/PN: " to the first line
>>
>> In some cases of long package names, this may be in conflict with
>> the first item.
>> "dev-python/rax-default-network-flags-python-novaclient-ext: " as a
>> prefix doesn't leave much space for an explanation. ;)

> I'd say that's rare enough to warrant exceptions in that case. Or
> are you suggesting to drop CATEGORY/PN completely? I mean... it's
> not strictly necessary, since you can just look at the files that
> have been touched and run git log on an ebuild directory, but I
> think it will still make using the history easier, especially since
> mass-commits will not have CATEGORY/PN and can very easily be
> identified. It's also common practice in various overlays.

> Do you want me to add this exceptional case as a valid reason to
> have more than 75 chars in the first line?

Everything else, like abbreviating PN, or moving it to third or later
line, doesn't seem reasonable. So, I'd go for exceptions to the 75
char rule.

>> | • for commits that affect only licenses directory, prepend
>> |   "licenses: " to the first line
>>
>> Same for commits that affect licenses and profiles?

> You are probably referring to license_groups.

Right.

> I'd say it matters more what the intention is and not so much what
> directories in particular were touched. If you want to add a new
> license and have to edit profiles/license_groups, I'd still just
> prepend "licenses: ".

> Maybe I should add this as a general idea to the commit guideline.
> I think that's better than specing the whole through which will
> probably just confuse people, no?

Sounds good. Just say that the summary should be prepended by an
indicator what part of the tree is (mainly) affected, followed by some
examples that need not cover all corner cases.

>> All in all, this looks sane to me. Is it planned to enforce this
>> policy by repoman?

> Good question. I'm not sure if it's a good idea though. It might get
> in our way for corner cases and whatnot. Maybe as a polite warning
> it would be ok.

Yes, I meant adding a warning, to make devs aware that there may be a
problem. A not quite perfect commit message is no real breakage, so it
doesn't merit a fatal error.

Ulrich


pgpRV6A2Xb4gB.pgp
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Patrick Lauer
On Monday 22 September 2014 00:52:14 hasufell wrote:
> > | • repoman must be run from all related ebuild directories (or
> > | 
> > |   related category directories or top-level directory) on the tip of
> > |   the local master branch (as in: right before you push and also
> > |   after resolving push-conflicts)
> > 
> > Have you tested if running repoman in the top-level directory is
> > realistic as part of the workflow?
> 
> This is really just meant for stuff like mass commits, not regular
> ebuild stuff.
> Ask patrick, he's running repoman all the time, probably even top-level.

It's not. For a category like dev-python I'm seeing runtimes near 10 minutes 
on a 3.4Ghz Xeon. Scaling that down to more common hardware it's even 
prohibitive on smaller categories ... and at our current commit frequency, 
mwehehehehe. hehe. Heh. It'd be an infinite pull-rebase-repoman-yell cycle.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Rich Freeman
On Sun, Sep 21, 2014 at 9:08 PM, Peter Stuge  wrote:
> hasufell wrote:
>> > A version bump plus cleaning up older ebuilds will be considered
>> > one logical change, I suppose?
>>
>> I'd consider it two logical changes
> ..
>> But I don't have a strong opinion on that
>
> I do - I think this is really important. Having clean history makes a
> huge difference for anyone who wants to use that history.
>
> One argument against those clean professional development practices
> that comes up over and over is that it takes more time, ("mimimi I
> don't have time to be part of any solution") which is sometimes true
> - but since git makes committing so easy usually the difference
> isn't very big, and the payoff when you benefit in the future is
> quite significant.

++

A git commit is virtually instantaneous since it is entirely local.

>
>
>> Do you think this should be added explicitly?
>
> I think keeping rules vague is probably the only thing that somehow
> scales.
>

++

I think we should start out with decent guidelines, and then move on from there.

Nobody is going to die if some of our commits are sloppy out of the
gate.  One of our biggest strengths as a distro is the autonomy we
give individual developers, and guidelines are usually more productive
than rules.

If they get abused, we can deal with it.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Peter Stuge
hasufell wrote:
> > A version bump plus cleaning up older ebuilds will be considered
> > one logical change, I suppose?
> 
> I'd consider it two logical changes
..
> But I don't have a strong opinion on that

I do - I think this is really important. Having clean history makes a
huge difference for anyone who wants to use that history.

One argument against those clean professional development practices
that comes up over and over is that it takes more time, ("mimimi I
don't have time to be part of any solution") which is sometimes true
- but since git makes committing so easy usually the difference
isn't very big, and the payoff when you benefit in the future is
quite significant.


Of course, lots of people still do not care at all about what is only
a potential benefit in an uncertain future. Personally I might prefer
that they stop doing open source instead of wasting my time with
their whining. You pay forward, that's the point.


Getting back on track, it's likely that first-time git users will
have to get used to committing more often than with other VCSes.


> and I'm not sure if we can enforce commit rules in such
> fine-grained details, can we?

It's impossible to catch all cases, and there will always be
disagreement between individual developers as to what is actually
appropriate useful and correct. I think explicit consensus is
impractical. :\


> Do you think this should be added explicitly?

I think keeping rules vague is probably the only thing that somehow
scales.

I really don't know. I get a feeling that a given individual either
understands this concept of atomic changes, or they don't. I haven't
seen someone who at first didn't understand (with someone explaining
it to them of course) it come around and "get it" sometime later. :(


//Peter



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread hasufell
Ulrich Mueller:
>> On Sun, 21 Sep 2014, hasufell  wrote:
> 
>> https://wiki.gentoo.org/wiki/Gentoo_git_workflow
> 
>> But so far, not many people have been particularly interested in the
>> details of these things. I'm also not sure if the ML is the right
>> way to figure out these details.
> 
> Where else should this be discussed then? Discussion page of the wiki
> page?
> 
> | commit policy
> | • atomic commits (one logical change)
> 
> A version bump plus cleaning up older ebuilds will be considered one
> logical change, I suppose?
> 

I'd consider it two logical changes (e.g. imagine a user complaining
about ebuild removal... you cannot easily revert it if it's not a
separate commit). But I don't have a strong opinion on that and I'm not
sure if we can enforce commit rules in such fine-grained details, can we?

Do you think this should be added explicitly?

> | • commits may span across multiple ebuilds/directories if it's one
> |   logical change 
> | • every commit on the left-most line of the history (that is, all
> |   the commits following the first parent of each commit) must be gpg
> |   signed by a gentoo dev 
> | • repoman must be run from all related ebuild directories (or
> |   related category directories or top-level directory) on the tip of
> |   the local master branch (as in: right before you push and also
> |   after resolving push-conflicts)
> 
> Have you tested if running repoman in the top-level directory is
> realistic as part of the workflow?
> 

This is really just meant for stuff like mass commits, not regular
ebuild stuff.
Ask patrick, he's running repoman all the time, probably even top-level.

> | commit message format
> | • all lines max 70-75 chars
> | • first line brief explanation
> | • second line always empty
> | • optional detailed multiline explanation must start at the third
> |   line
> | • for commits that affect only a single package, prepend
> |   "CATEGORY/PN: " to the first line
> 
> In some cases of long package names, this may be in conflict with the
> first item. "dev-python/rax-default-network-flags-python-novaclient-ext: "
> as a prefix doesn't leave much space for an explanation. ;)
> 

I'd say that's rare enough to warrant exceptions in that case. Or are
you suggesting to drop CATEGORY/PN completely? I mean... it's not
strictly necessary, since you can just look at the files that have been
touched and run git log on an ebuild directory, but I think it will
still make using the history easier, especially since mass-commits will
not have CATEGORY/PN and can very easily be identified. It's also common
practice in various overlays.

Do you want me to add this exceptional case as a valid reason to have
more than 75 chars in the first line?

> | • for commits that affect only a single package, but also modify
> |   eclasses/profiles/licenses as part of a logical change, also
> |   prepend "CATEGORY/PN: " to the first line
> | • for commits that affect only the profile directory, prepend
> |   "profiles: " to the first line
> | • for commits that affect only the eclass directory, prepend
> |   "ECLASSNAME.eclass: " to the first line
> 
> Maybe just "eclass: " would be slightly more systematic? (Because for
> all others it is a directory.)
> 

Not sure either. This is again just a convenience thing for history
readability, not strictly necessary information. But I'd personally go
for full eclass name.

> | • for commits that affect only licenses directory, prepend
> |   "licenses: " to the first line
> 
> Same for commits that affect licenses and profiles?
> 

You are probably referring to license_groups. I'd say it matters more
what the intention is and not so much what directories in particular
were touched. If you want to add a new license and have to edit
profiles/license_groups, I'd still just prepend "licenses: ".

Maybe I should add this as a general idea to the commit guideline. I
think that's better than specing the whole through which will probably
just confuse people, no?

> | • for commits that affect only metadata directory, prepend
> |   "metadata: " to the first line
> | • mass commits that affect a whole category (or large parts of it)
> |   may prepend "CATEGORY: " to the first line
> 
> All in all, this looks sane to me. Is it planned to enforce this
> policy by repoman?
> 

Good question. I'm not sure if it's a good idea though. It might get in
our way for corner cases and whatnot. Maybe as a polite warning it would
be ok.


Also, if you have ideas of wording something better in particular,
please share or just edit the wiki.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Ulrich Mueller
> On Sun, 21 Sep 2014, Peter Stuge wrote:

> Jonathan Callen wrote:
>> the correct response would be to ensure that the final commit
>> pushed (whether it be a merge commit or rebased) contains the
>> stabilization for both arches

> I think this is one of the things to check in a post-receive or
> post-update hook. What is the easiest way to access keywords out of
> an ebuild - which does *not* require sourcing the ebuild in a shell?

A quick scan of the tree shows that combinations of any of the
following would have to be handled:
- leading whitespace
- double or single quotes, or no quotation marks at all
- continuation lines, with or without backslash
- comments following the keywords assignment
- keywords inherited from an eclass

Of course, there is no guarantee that devs don't invent something new
that isn't in above list. :) So I'd say there is no sane way other
than sourcing with bash, unless we want to restrict by policy what is
allowed as KEYWORDS syntax.

Ulrich


pgp8u_OncpcuM.pgp
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Peter Stuge
Jonathan Callen wrote:
> the correct response would be to ensure that the final commit
> pushed (whether it be a merge commit or rebased) contains the
> stabilization for both arches

I think this is one of the things to check in a post-receive or
post-update hook. What is the easiest way to access keywords out of
an ebuild - which does *not* require sourcing the ebuild in a shell?


//Peter



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Jonathan Callen
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 09/21/2014 01:21 PM, Michał Górny wrote:
> Dnia 2014-09-18, o godz. 19:39:08 Tobias Klausmann
>  napisał(a):
> 
>> Since we're causing at least mild upheaval process-wise, I 
>> thought I'd bring up a topic that will be exacerbated by the git 
>> migration if it's not really addressed.
>> 
>> AIUI, we try to avoid merge conflicts, unless the merge is a 
>> meaningful integration of divergent processes.
>> 
>> However, one aspect of how ebuilds are written these days will 
>> cause a non-trivial amount of merge commits that are not
>> actually useful in that sense.
>> 
>> This is due to the way keywording and stabilization work on an 
>> ebuild level. Since keywords are all in one line, any merge tool 
>> will barf on two keywords being changed in disparate clones.
>> I.e. if I change ~alpha->alpha while someone else changes 
>> ~amd64->amd64, we will have a merge conflict.
> 
> If someone stabilizes the package you have edited, then most likely
> you actually want to edit your commits and move the changes to a
> revbump.
> 
> If at all, I'd be more worried by a case when queued version bumps 
> would lose keywords that were added in the meantime to older
> versions.
> 

The case under discussion here was where the only edit made locally to
the ebuild was, in fact, stabilizing the ebuild on a different
architecture.  In this case, the correct response would be to ensure
that the final commit pushed (whether it be a merge commit or rebased)
contains the stabilization for both arches, as there would be no need
to revbump to add a stable keyword (in fact, I'd call that an
incorrect resolution).

- -- 
Jonathan Callen
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQIcBAEBCgAGBQJUHy8BAAoJELHSF2kinlg4wPIP/jrDAo0ye4f/5jRF3AwZYTN8
UkEJywDWFA4K9ffZbuAN2gGEWGclh6p5fMiiPwKjxAaMDzopuWdzWRrVYXsTEqqW
SKT0jmAQroSiXj3DLczuvijw0xhY1sKLAUUnnsxhdieYb61SusIBV5xtDl4R/xuD
pF8wtlEp/KL3Kbba4wlcBL4lZsSannuNZ8BA0QyGhPfiLb/UEYQsL+hhwWNR9IlM
HnDuBzdHFvJP29QfP556+ItxoKFDFJpRMYqukN2Ws1g276DesryWe0iWF9hvzCyj
VdTml7T+u1i7VXM9UCE0IAp7r94y/14Q1S/+XWLNxATFxI2xVP3HuecZVKwDYhcy
zu8FTcRmYZddIzhcnUKVWe/e5ihIvYlesEoKVgsl6TCXbcbshzyKEpfZXcjTs5Po
gQe32QPDwLetqO5qbko51SbO2E1gCMEFrzf+MsBLE2oO+nhfEbL6Gmsx5F7beM37
AafEdb8U3ZBi7jz2/zPGBKzoyDinbjB9o5I+mrwG7qL4t0OZDjS9Qg/eOg3lm3cF
5tQbMNvVOzxBocis3FSk+L9lMVxQKQAYCrYn52meZbbCdFc85bnqXuCZvh1Q+mp0
zgX8vroDDyzpzP6AJeq8haP10TExnVABbsZHV9YxhVEipQO0mPrtzDWodS157dEN
yzgJrofBkxqf/r2hUcvX
=gARN
-END PGP SIGNATURE-



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Ulrich Mueller
> On Sun, 21 Sep 2014, hasufell  wrote:

> https://wiki.gentoo.org/wiki/Gentoo_git_workflow

> But so far, not many people have been particularly interested in the
> details of these things. I'm also not sure if the ML is the right
> way to figure out these details.

Where else should this be discussed then? Discussion page of the wiki
page?

| commit policy
| • atomic commits (one logical change)

A version bump plus cleaning up older ebuilds will be considered one
logical change, I suppose?

| • commits may span across multiple ebuilds/directories if it's one
|   logical change 
| • every commit on the left-most line of the history (that is, all
|   the commits following the first parent of each commit) must be gpg
|   signed by a gentoo dev 
| • repoman must be run from all related ebuild directories (or
|   related category directories or top-level directory) on the tip of
|   the local master branch (as in: right before you push and also
|   after resolving push-conflicts)

Have you tested if running repoman in the top-level directory is
realistic as part of the workflow?

| commit message format
| • all lines max 70-75 chars
| • first line brief explanation
| • second line always empty
| • optional detailed multiline explanation must start at the third
|   line
| • for commits that affect only a single package, prepend
|   "CATEGORY/PN: " to the first line

In some cases of long package names, this may be in conflict with the
first item. "dev-python/rax-default-network-flags-python-novaclient-ext: "
as a prefix doesn't leave much space for an explanation. ;)

| • for commits that affect only a single package, but also modify
|   eclasses/profiles/licenses as part of a logical change, also
|   prepend "CATEGORY/PN: " to the first line
| • for commits that affect only the profile directory, prepend
|   "profiles: " to the first line
| • for commits that affect only the eclass directory, prepend
|   "ECLASSNAME.eclass: " to the first line

Maybe just "eclass: " would be slightly more systematic? (Because for
all others it is a directory.)

| • for commits that affect only licenses directory, prepend
|   "licenses: " to the first line

Same for commits that affect licenses and profiles?

| • for commits that affect only metadata directory, prepend
|   "metadata: " to the first line
| • mass commits that affect a whole category (or large parts of it)
|   may prepend "CATEGORY: " to the first line

All in all, this looks sane to me. Is it planned to enforce this
policy by repoman?

Ulrich


pgpuJcrBWOtmG.pgp
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Michał Górny
Dnia 2014-09-18, o godz. 19:39:08
Tobias Klausmann  napisał(a):

> Since we're causing at least mild upheaval process-wise, I
> thought I'd bring up a topic that will be exacerbated by the git
> migration if it's not really addressed.
> 
> AIUI, we try to avoid merge conflicts, unless the merge is a
> meaningful integration of divergent processes.
> 
> However, one aspect of how ebuilds are written these days will
> cause a non-trivial amount of merge commits that are not actually
> useful in that sense.
> 
> This is due to the way keywording and stabilization work on an
> ebuild level. Since keywords are all in one line, any merge tool
> will barf on two keywords being changed in disparate clones. I.e.
> if I change ~alpha->alpha while someone else changes
> ~amd64->amd64, we will have a merge conflict.

If someone stabilizes the package you have edited, then most likely you
actually want to edit your commits and move the changes to a revbump.

If at all, I'd be more worried by a case when queued version bumps
would lose keywords that were added in the meantime to older versions.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread hasufell
Tobias Klausmann:
> When we do the migration, there _will_ be
> confusion and breakage and those who actually have deep knowledge
> will likely cringe a lot. Documentation is the way out of that.
> 

https://wiki.gentoo.org/wiki/Gentoo_git_workflow

But so far, not many people have been particularly interested in the
details of these things. I'm also not sure if the ML is the right way to
figure out these details.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-21 Thread Tobias Klausmann
Hi! 

On Fri, 19 Sep 2014, hasufell wrote:
> Tobias Klausmann:
> >>> If this should really turn out to be a problem, then we could also:
> >>>
> >>> 4) Replace git's default merge driver by our own one that is better
> >>> suited for ebuilds. This can be done per repository via .git/config
> >>> and .gitattributes.
> >>
> >> Certainly that would be even more helpful!
> > 
> > Still, all of these scenarios cause merge commits
> 
> No.
> 
> 1. git pull --rebase=preserve origin master
> => error: could not apply ... 
> 2. fix conflicts via 'git mergetool' (e.g. meld or vimdiff with 3 panel
> view... very easy to see what happened)
> 3. finish rebase via 'git rebase --continue'
> => your unpushed keyword commit has been rewritten without a merge commit
> 4. push

See, this is why I asked: I was not aware of this (and have
pointed out repeatedly that I'd be delighted to be educated).

> That is pretty easy and takes you ~20s for a keyword merge.
> What's the problem?

The problem is that not everyone has deep knowledge of git. I
asked because I wanted to know what (if anything) we can do about
a problem I perceived. When we do the migration, there _will_ be
confusion and breakage and those who actually have deep knowledge
will likely cringe a lot. Documentation is the way out of that.

Regards,
Tobias

-- 
printk("Penguin %d is stuck in the bottle.\n", i);
linux-2.0.38/arch/sparc/kernel/smp.c



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-20 Thread hasufell
Rich Freeman:
> 
> Would it make more sense to use a migrated repository as the starting
> point, such as this one:
> https://github.com/rich0/gentoo-gitmig-2014-09-15
> 

Not sure why. The old history is irrelevant for testing git workflow.

The repository is also fairly small, even without shallow clone (and the
git version that supports pushing from shallow clones isn't in stable
arch yet).

> Also, can you grant me access to push to gentoo/gentoo-gitmig?
> 

You are not in the gentoo group on github? Everyone within that group
should have push access.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-20 Thread Rich Freeman
On Sat, Sep 20, 2014 at 10:28 AM, hasufell  wrote:
> Ian Stakenvicius:
>>> I wonder if it would make sense to set up a practice git tree
>>> somewhere so that people can try working together on workflows/etc.
>>> We can clone a migrated tree (I have one from a few days ago on
>>> github).
>>
>> Definitely.  I'd volunteer for that (doing my updates to both git and
>> cvs trees), and I expect others would as well.
>>
>
>
> https://github.com/gentoo/gentoo-gitmig
>
> * the cloned repo is about ~600mb big after checking out the files (the
> .git dir is just 76mb) and has no significant history, so a normal clone
> fetches about the same size of a portage snapshot
> * last import of ebuilds was today (and I think I'll leave it at that)
> * no ChangeLogs
> * thin Manifests
>

Would it make more sense to use a migrated repository as the starting
point, such as this one:
https://github.com/rich0/gentoo-gitmig-2014-09-15

Also, can you grant me access to push to gentoo/gentoo-gitmig?

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-20 Thread hasufell
Ian Stakenvicius:
>> I wonder if it would make sense to set up a practice git tree 
>> somewhere so that people can try working together on workflows/etc.
>> We can clone a migrated tree (I have one from a few days ago on
>> github).
> 
> Definitely.  I'd volunteer for that (doing my updates to both git and
> cvs trees), and I expect others would as well.
> 


https://github.com/gentoo/gentoo-gitmig

* the cloned repo is about ~600mb big after checking out the files (the
.git dir is just 76mb) and has no significant history, so a normal clone
fetches about the same size of a portage snapshot
* last import of ebuilds was today (and I think I'll leave it at that)
* no ChangeLogs
* thin Manifests



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Ulrich Mueller
> On Fri, 19 Sep 2014, hasufell  wrote:

> There is no reason to roll back commits or do merge commits for the
> keywords conflict use case. So I don't see what else needs
> discussion here, except the proposal from ulm.

About an optimised merge driver for ebuilds? Please don't see this as
a blocker for the migration.

Ulrich


pgpVH2cxxftNs.pgp
Description: PGP signature


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread hasufell
Ian Stakenvicius:
> Git on the other hand will update the entire
> tree and there's no way around that, right?

Yeah, people have to understand that we cannot map the cvs workflow 1:1
to git.

That's for a reason and that little inconvenience it causes is pretty
minor compared to the breakage CVS allows (user syncs at a bad time and
gets a broken tree, repoman gives bogus results etc.).

There is no reason to roll back commits or do merge commits for the
keywords conflict use case. So I don't see what else needs discussion
here, except the proposal from ulm.



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Ian Stakenvicius
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 19/09/14 11:10 AM, Ian Stakenvicius wrote:
> 
> I don't think there's any valid debate on whether git is more 
> efficient than cvs on fetching tree-wide updates :)
> 

erm, that was worded wrong, let me try again:  I don't think there's
any valid debatable point against git being more efficient than cvs,
when fetching updates tree-wide.
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iF4EAREIAAYFAlQcSCoACgkQ2ugaI38ACPCU1QD8Dqy00yazdlXLphGYKx4e/Ec6
ci9AyvPI3ga+ilqmaJUA/RlsRM6ycPr6jXaW5WZtPTxjgfuz0qSrsw2cf+K7UjGK
=p9XG
-END PGP SIGNATURE-



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Ian Stakenvicius
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 19/09/14 10:48 AM, Rich Freeman wrote:
> On Fri, Sep 19, 2014 at 10:25 AM, hasufell  
> wrote:
>> 
>> That is pretty easy and takes you ~20s for a keyword merge. 
>> What's the problem?
>> 
> 
> Agree.  Also, there was a comment that git pull is much slower than
> cvs.  While it is true that git does refresh the whole tree all the
> time, it is FAR more efficient at doing this than CVS, since the
> local and remote repositories can use a single hash to determine
> where each stands relative to the other, and the COW mechanism
> applies to directory trees so when making comparisons git does not
> need to traverse the full depth of the tree for every branch.  A
> cvs update on the entire tree is basically an independent
> synchronization of every file in the tree.  Also, by refreshing the
> entire tree you also catch any repoman errors that might result
> from commits to other packages that you didn't have visibility to
> when refreshing only a single package in cvs.
> 

That wasn't really part of his argument, though -- when he's fixing
keyword collisions, he's only working within one package, and CVS in
that case -only- checks the subtree as of that package (ie, ./ and
./files/) when updating.  Git on the other hand will update the entire
tree and there's no way around that, right?  (unless of course i
missed something in hasufell's command list)

I don't think there's any valid debate on whether git is more
efficient than cvs on fetching tree-wide updates :)


> I wonder if it would make sense to set up a practice git tree 
> somewhere so that people can try working together on workflows/etc.
> We can clone a migrated tree (I have one from a few days ago on
> github).

Definitely.  I'd volunteer for that (doing my updates to both git and
cvs trees), and I expect others would as well.


-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iF4EAREIAAYFAlQcR1UACgkQ2ugaI38ACPA1ogEAmn9ZBr9nORFH02cxu4HML+9C
OKyPystsxAaOdEnkzS8A/29mitUtED8jowJ+5Udh9YyYRjJApzx36hRMyyAsJUVc
=B0E9
-END PGP SIGNATURE-



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Rich Freeman
On Fri, Sep 19, 2014 at 10:25 AM, hasufell  wrote:
>
> That is pretty easy and takes you ~20s for a keyword merge. What's the
> problem?
>

Agree.  Also, there was a comment that git pull is much slower than
cvs.  While it is true that git does refresh the whole tree all the
time, it is FAR more efficient at doing this than CVS, since the local
and remote repositories can use a single hash to determine where each
stands relative to the other, and the COW mechanism applies to
directory trees so when making comparisons git does not need to
traverse the full depth of the tree for every branch.  A cvs update on
the entire tree is basically an independent synchronization of every
file in the tree.  Also, by refreshing the entire tree you also catch
any repoman errors that might result from commits to other packages
that you didn't have visibility to when refreshing only a single
package in cvs.

I wonder if it would make sense to set up a practice git tree
somewhere so that people can try working together on workflows/etc.
We can clone a migrated tree (I have one from a few days ago on
github).

I'm not super-familiar with github organizations, but if it is easy to
grant everybody in the gentoo organization access to a migrated tree
we could do that, and host it in the organization.  I don't care where
we host the tree - I just don't have anything set up personally.

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread hasufell
Tobias Klausmann:
>>>
>>> If this should really turn out to be a problem, then we could also:
>>>
>>> 4) Replace git's default merge driver by our own one that is better
>>> suited for ebuilds. This can be done per repository via .git/config
>>> and .gitattributes.
>>>
>>
>> Certainly that would be even more helpful!
> 
> Still, all of these scenarios cause merge commits

No.

1. git pull --rebase=preserve origin master
=> error: could not apply ... 
2. fix conflicts via 'git mergetool' (e.g. meld or vimdiff with 3 panel
view... very easy to see what happened)
3. finish rebase via 'git rebase --continue'
=> your unpushed keyword commit has been rewritten without a merge commit
4. push

That is pretty easy and takes you ~20s for a keyword merge. What's the
problem?



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Tobias Klausmann
Hi! 

On Fri, 19 Sep 2014, Tom Wijsman wrote:
> On Thu, 18 Sep 2014 19:39:08 +0200
> Tobias Klausmann  wrote:
> 
> > AIUI, we try to avoid merge conflicts, unless the merge is a
> > meaningful integration of divergent processes.
> > 
> > However, one aspect of how ebuilds are written these days will
> > cause a non-trivial amount of merge commits that are not actually
> > useful in that sense.
> 
> The concept of rebasing your commits has been invented for this. In the
> less common case that multiple people change the keywords, a manual
> three way merge is a breeze; taking into consideration that maintainers
> should be aware of KEYWORDS and other recent changes to their packages.

As I pointed out, getting the right code into the tree is not the
problem here. It is extra work over the current way of doing it
(since I need to deal with a local commit that can't be ff'd or
rebased as git is very line-oriented.

On top of the extra work, there have been several mentions that
only meaning ful merge-commits are wanted in the tree (or not
wanted at all). AIUI, avoiding them in the keywording/stabilizing
phase is currently very difficult, unless we split KEYWORDS into
separate lines or find another mechanism (like having the
maintainer/keyword-requestor do all the edits after the archs
sign off on them).

I'd be delighted to hear a simpler solution that only involves
doing the semantic merge (like we do now with CVS).

And at least in my case, collisions during keywording are fairly
common, and will be even more so with git since fetch/pull are
slower than for a CVS subdir (since git checks the whole tree for
local changes, not just the current subtree, AIUI).

Again, correct me if I'm wrong. I've been using git for ~4 years,
but only in single-developer mode. And even there I managed to
make a mess of repo histories :-/ Fortunately, nobody cares about
my stuff.

Regards,
Tobias



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Tom Wijsman
On Thu, 18 Sep 2014 19:39:08 +0200
Tobias Klausmann  wrote:

> AIUI, we try to avoid merge conflicts, unless the merge is a
> meaningful integration of divergent processes.
> 
> However, one aspect of how ebuilds are written these days will
> cause a non-trivial amount of merge commits that are not actually
> useful in that sense.

The concept of rebasing your commits has been invented for this. In the
less common case that multiple people change the keywords, a manual
three way merge is a breeze; taking into consideration that maintainers
should be aware of KEYWORDS and other recent changes to their packages.



Re: [gentoo-scm] Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Kent Fredric
On 19 September 2014 06:51, Rich Freeman  wrote:

>
> Git even allows the use of tools like meld to resolve conflicts,
> besides the usual fill-the-file-with-diffs-to-cleanup approach.


I'd personally discourage any kind of collision resolution in the merge
itself.

Mostly because I've been bitten by attempted rebases where those collision
resolutions have been *lost* .

Much preferable to rebase one branch or find some otherway to weed the
collision out prior to merge and giving a clean track record of where the
fix was applied. ( Because the fix happening in the merge is a great way to
hide accidental code changes )


-- 
Kent

*KENTNL* - https://metacpan.org/author/KENTNL


Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-19 Thread Tobias Klausmann
Hi! 

On Thu, 18 Sep 2014, Rich Freeman wrote:
> On Thu, Sep 18, 2014 at 1:53 PM, Ulrich Mueller  wrote:
> >> On Thu, 18 Sep 2014, Tobias Klausmann wrote:
> >
> >> However, one aspect of how ebuilds are written these days will
> >> cause a non-trivial amount of merge commits that are not actually
> >> useful in that sense.
> >
> 
> Git can do merge conflict markers just like cvs.  Also, in practice I
> don't tend to run into merge conflicts with cvs - I always do a
> directory update before making changes.  With git I'd probably not do
> a pull if I were working on an obscure pacakge, but if I were doing a
> security keywording I'd certainly do a pull before touching anything
> since the likelihood of a conflict is much higher.

The problem isn't the conflict markers. The problem is that with
git, by the time I have a conflict, I also have a local commit
that I have to roll back or turn into a merge, which means extra
work.

> Git even allows the use of tools like meld to resolve conflicts,
> besides the usual fill-the-file-with-diffs-to-cleanup approach.
> 
> >
> > If this should really turn out to be a problem, then we could also:
> >
> > 4) Replace git's default merge driver by our own one that is better
> > suited for ebuilds. This can be done per repository via .git/config
> > and .gitattributes.
> >
> 
> Certainly that would be even more helpful!

Still, all of these scenarios cause merge commits, which some
people seem to be rather allergic to (and I do agree that nothing
of use is actually merged, since two stabilizations/keywordings
are usually orthogonal).

Regards,
Tobias



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-18 Thread Rich Freeman
On Thu, Sep 18, 2014 at 1:53 PM, Ulrich Mueller  wrote:
>> On Thu, 18 Sep 2014, Tobias Klausmann wrote:
>
>> However, one aspect of how ebuilds are written these days will
>> cause a non-trivial amount of merge commits that are not actually
>> useful in that sense.
>

Git can do merge conflict markers just like cvs.  Also, in practice I
don't tend to run into merge conflicts with cvs - I always do a
directory update before making changes.  With git I'd probably not do
a pull if I were working on an obscure pacakge, but if I were doing a
security keywording I'd certainly do a pull before touching anything
since the likelihood of a conflict is much higher.

Git even allows the use of tools like meld to resolve conflicts,
besides the usual fill-the-file-with-diffs-to-cleanup approach.

>
> If this should really turn out to be a problem, then we could also:
>
> 4) Replace git's default merge driver by our own one that is better
> suited for ebuilds. This can be done per repository via .git/config
> and .gitattributes.
>

Certainly that would be even more helpful!

--
Rich



Re: [gentoo-dev] Gentoo git workflows and the stabilization/keywording process

2014-09-18 Thread Ulrich Mueller
> On Thu, 18 Sep 2014, Tobias Klausmann wrote:

> However, one aspect of how ebuilds are written these days will
> cause a non-trivial amount of merge commits that are not actually
> useful in that sense.

> This is due to the way keywording and stabilization work on an
> ebuild level. Since keywords are all in one line, any merge tool
> will barf on two keywords being changed in disparate clones. I.e.
> if I change ~alpha->alpha while someone else changes
> ~amd64->amd64, we will have a merge conflict.

> There several approaches to this problem:

> 1) Make a merge commit as explained above. Aside from the
> not really useful merge commit, this also means manual work for
> the ATs, who could really do with less manual labor.

> 2) roll back the local commit, fetch and re-do the semantically
> equivalent commit. This might be automated, but someone has to
> actually write that code.

> 3) Do away with one-line KEYWORD lists. This has wide-ranging
> repercussions, from grepping-and-parsing (which is already broken
> in different ways[0]), and it makes the "preamble" of ebuilds
> longer.

If this should really turn out to be a problem, then we could also:

4) Replace git's default merge driver by our own one that is better
suited for ebuilds. This can be done per repository via .git/config
and .gitattributes.

Ulrich


pgp6wMLsX3Mtj.pgp
Description: PGP signature