Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-02 Thread Rich Freeman
On Sat, Jun 2, 2012 at 12:04 AM, Robin H. Johnson robb...@gentoo.org wrote:
 please look up git-bundle before suggesting things like tarballs of
 repos/checkouts.


Looks useful.  Wasn't aware that a bundle was something other than a tarball.

We'll probably need to spell out the preferred process in the docs,
and reference it frequently in communications.  Otherwise you'll get
quite a few clones instead.

It appears that devs will have to add the remote for the live
repository after they've cloned the bundle - otherwise they'll just
keep pulling from the bundle which isn't all that convenient.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-02 Thread Dirkjan Ochtman
On Sat, Jun 2, 2012 at 12:59 PM, Rich Freeman ri...@gentoo.org wrote:
 Looks useful.  Wasn't aware that a bundle was something other than a tarball.

 We'll probably need to spell out the preferred process in the docs,
 and reference it frequently in communications.  Otherwise you'll get
 quite a few clones instead.

 It appears that devs will have to add the remote for the live
 repository after they've cloned the bundle - otherwise they'll just
 keep pulling from the bundle which isn't all that convenient.

I think you still misunderstand. As I understand Robin, we wouldn't
even offer up a clone of the full-history bundle, it would only be
offered as a normal download. The default workflow is cloning from the
shallow version, which will obviously give you the desired remote.

Cheers,

Dirkjan



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-02 Thread Rich Freeman
On Sat, Jun 2, 2012 at 8:03 AM, Dirkjan Ochtman d...@gentoo.org wrote:
 On Sat, Jun 2, 2012 at 12:59 PM, Rich Freeman ri...@gentoo.org wrote:
 It appears that devs will have to add the remote for the live
 repository after they've cloned the bundle - otherwise they'll just
 keep pulling from the bundle which isn't all that convenient.

 I think you still misunderstand. As I understand Robin, we wouldn't
 even offer up a clone of the full-history bundle, it would only be
 offered as a normal download. The default workflow is cloning from the
 shallow version, which will obviously give you the desired remote.

I wasn't talking about full-history.

I was talking about the fact that we're distributing a bundle.  If you
clone a bundle, you won't have a remote for the live repository.  You
would need to add it, unless you plan to never push a commit back to
the gentoo repository, and you plan to manually download bundles
anytime you want to update your local repository.

I'm not sure how exactly Robin was planning on making the full history
available, but it sounded like it would also be distributed as a
bundle.  That means that you can certainly clone it - just type git
clone path-to-locally-saved-bundle-file .  If it is in some other
format like a pack file then you would import it into a repository via
a different command.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Dirkjan Ochtman
On Thu, May 31, 2012 at 6:49 PM, Robin H. Johnson robb...@gentoo.org wrote:
 Discussion on merge policy. Originally I thought we would disallow merge
 commits, so that we would get a cleaner history. However, it turns out that if
 the repo ends up being pushed to different places with slightly different
 histories, merges are absolutely going to be required to prevent somebody from
 having to rebase at least one of their sets of commits that are already 
 pushed.

Can you elaborate on why the cleaner history a no-merge policy
enforces is a good thing? I actually think that seeing merge commits
might clarify the history; it can be valuable to see that some mistake
was made in a merge instead, but you can only see that if there's an
explicit merge commit.

I should note that I come at this from the Mercurial side, where the
immutability of (public) history has historically carried more value
than on the git side (and related to that, rebase-like tools were less
integrated until relatively recently).

Cheers,

Dirkjan



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 1 June 2012 20:16, Dirkjan Ochtman d...@gentoo.org wrote:
 Can you elaborate on why the cleaner history a no-merge policy
 enforces is a good thing? I actually think that seeing merge commits
 might clarify the history; it can be valuable to see that some mistake
 was made in a merge instead, but you can only see that if there's an
 explicit merge commit.

 I should note that I come at this from the Mercurial side, where the
 immutability of (public) history has historically carried more value
 than on the git side (and related to that, rebase-like tools were less
 integrated until relatively recently).

 Cheers,

 Dirkjan


One of the perk of fast-forward only histories is merge problems don't happen.

If a commit sequence isn't fast-fowardable, then the submitter has to
rebase it, and a rebased history is a guaranteed clean merge.

It also puts the onus on making the patch sequence mergable on the
contributor of that patch sequence, by forcing them to resolve
conflicts before they can submit their branch for inclusion, and this
is good, because they understand their own changes best, and are the
best person to decide how to deal with conflicts.

The best example of this is 2 people contribute behaviourally
identical code, with different syntax .

In the case of a merge, the person performing the merge has to decide
which one to take, and they might not be the best person to do so. (
And historically, it can be confusing if you look at the parents of
the merge, and find the 2 competing implementations, of which only 1
is relevant )

If you request that a branch is rebased before it is integrated, then
this problem solves itself in a way.

Whoevers implementation gets in the tree first gets in, and then the
second person rebases their branch on top of that.

In the process of performing the rebase, the second person will
discover the precise commit on their side that introduces a conflict,
and be able to decide if they need to replace the existing code
anyway, or if they need to destroy their own code.

And when they're done, their entire commit series should be sane and
logical as if the first persons code was there in the first place
before the second person even started coding.

a---b---cd---e---f
    |  \|/
\---x---yz

Essentially, in this diagram, if d and y are different, but
equivalent, they will cause a collision when merge Z is performed.

By performing a rebase of  the second branch, the logical order becomes

a---b---cd---e---f---x---y---z

And when the rebaser is applying y, they will notice it is likely
redundant, and the history will become

a---b---cd---e---f---x---z

And in both the merge and rebase examples, z will be the same, just
the cognitive overhead of understanding what the hell happened in
the latter case is simpler, as the biggest delta between any 2 commits
will be 1 commit worth, whereas in the merge case,  there will be a
huge delta  between the commit before the merge, and the commit after
the merge, and looking at the merge itself diff-wise, you'll be
comparing the aggregate result of 2 entire branches worth of commits,
as opposed to only comparing one small and simple commit with another.


-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 12:05 AM, Michał Górny mgo...@gentoo.org wrote:
 Yes, it isn't but such kind of work flow was presented in the message I
 replied to.

Yup, I wasn't aware that when rebasing you have the option to squash
commits or not.  They all get rewritten as if they were applied to the
current head in the first place, but apparently unless you squash them
you still get all the individual commits.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 12:55 AM, Kent Fredric kentfred...@gmail.com wrote:

 Hmm, thats annoying. Almost makes me wish it was the trees that were
 signed, not the commits.

I think it is the tree that is signed, but that changes too.

Rebasing re-applies the same diff to the new head to give you a new
set of commits.  When you apply the same diff to a different parent
you end up with a different tree, so the tree signature won't be the
same either.

Keep in mind that git does not store a long train of diffs.  It stores
a long chain of complete trees, and the diffs get calculated if you
ask for them.  Since it is COW you only re-store files that actually
change, and incorporate others by reference.  However, if you have a
1MB file that you change 1 line on 100x, you'll end up with 100MB of
files.  Of course, when they get packed I'd hope that they'd compress
well.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 1 June 2012 22:54, Rich Freeman ri...@gentoo.org wrote:
 On Fri, Jun 1, 2012 at 12:55 AM, Kent Fredric kentfred...@gmail.com wrote:

 Hmm, thats annoying. Almost makes me wish it was the trees that were
 signed, not the commits.

 I think it is the tree that is signed, but that changes too.

Nope, at least not as far as I can tell, and I just implemented commit
signature verification _

 Rebasing re-applies the same diff to the new head to give you a new
 set of commits.  When you apply the same diff to a different parent
 you end up with a different tree, so the tree signature won't be the
 same either.

Not nessecarily. Given that :

 a file with a given content has a fixed SHA
A tree is just a list of these SHA's , and that in turn is referenced
by SHA, so if 2 commits have identical file content, their 'tree' sha
will be the same ( in theory ).

So that means, if you perform a rebase, assuming the filesystem looks
the same as it did before the rebase, it will have the same SHA1 for
the tree, regardless of the process of how it got to be that way.

The only SHA that has to change is the 'parent',

( Demonstration here: https://gist.github.com/2851330 , note I have 2
commits with the same tree sha, and the tree sha only really refers to
one file 3 times as all the empty files have same sha  )

But unfortunately, with a rebase, even if the trees don't change, if
the history order changes, the commits themselves have to change due
to the parent sha needing to change ( yes, I know commits are
immutable in reality, and they don't change, but are duplicated and
the duplicate is given the new sha , but the effect is still the same
, because those unreferenced commits will eventually get reaped )


-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Michał Górny
On Fri, 1 Jun 2012 23:23:34 +1200
Kent Fredric kentfred...@gmail.com wrote:

 On 1 June 2012 22:54, Rich Freeman ri...@gentoo.org wrote:
  Rebasing re-applies the same diff to the new head to give you a new
  set of commits.  When you apply the same diff to a different parent
  you end up with a different tree, so the tree signature won't be the
  same either.
 
 Not nessecarily. Given that :
 
  a file with a given content has a fixed SHA
 A tree is just a list of these SHA's , and that in turn is referenced
 by SHA, so if 2 commits have identical file content, their 'tree' sha
 will be the same ( in theory ).
 
 So that means, if you perform a rebase, assuming the filesystem looks
 the same as it did before the rebase, it will have the same SHA1 for
 the tree, regardless of the process of how it got to be that way.

I don't think that 'not necessarily' makes any difference here. Maybe
in our particular case this is not as likely as with regular source
code tree but while rebasing you can hit conflicts. And then files
start to change...

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric kentfred...@gmail.com wrote:

 Nope, at least not as far as I can tell, and I just implemented commit
 signature verification _

I've been trying to find an example of a signed commit, but can't find
one anywhere, so it is hard to tell what it is doing without going
through the git source carefully.  If you have an example of a signed
commit feel free to send it to me.

Note that I am NOT interested in the output of any git operation (such
as git log --show-signature, git show, etc) - these are all processed
and do not reveal what is actually going on.  I want a copy of the
actual file containing the signature.  If the signature is embedded in
the commit then I want the file in the objects directory tree that
contains the commit.  They're just compressed text files (though it is
a bit of a pain to decompress them).

 Not nessecarily. Given that :

  a file with a given content has a fixed SHA
 A tree is just a list of these SHA's , and that in turn is referenced
 by SHA, so if 2 commits have identical file content, their 'tree' sha
 will be the same ( in theory ).

 So that means, if you perform a rebase, assuming the filesystem looks
 the same as it did before the rebase, it will have the same SHA1 for
 the tree, regardless of the process of how it got to be that way.

The filesystem WON'T look the same after a rebase.

quick example (operations done in this order):

file in commit A in master:
1
2
3
4
5

file in commit B in a branch off of master:
1
2a
3
4
5

file in commit C in master:
1
2
3
4a
5

if you merge master into the branch you'll end up with a new commit D
whose parent is B that looks like:
1
2a
3
4a
5

if instead you rebase master into the branch you'll end up with a new
commit D whose parent is C that looks like:
1
2a
3
4a
5

The history for the branch will no longer contain B.  If there were 14
commits B1..14 you'd end up with 14 commits D1.14 that each contain
the line 4a.  None of them would use the same trees as B1..14, and
they can't use the same signatures as a result, even if only the tree
was signed.   As far as the new history was concerned, line 4a was
there before the branch was started.

At least, that is my understanding of rebasing.  Again, I'm a bit of a
git novice, but I never really grokked git until I saw a presentation
that didn't start with commands, but instead started out with just
walking through the actual structure of the repository.  Once you grok
the COW model I find it all clicks into place.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Alexey Shvetsov

Hi!

Check kde overlay ;) we used signed commits here

Rich Freeman писал 2012-06-01 16:42:
On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric kentfred...@gmail.com 
wrote:


Nope, at least not as far as I can tell, and I just implemented 
commit

signature verification _


I've been trying to find an example of a signed commit, but can't 
find

one anywhere, so it is hard to tell what it is doing without going
through the git source carefully.  If you have an example of a signed
commit feel free to send it to me.

Note that I am NOT interested in the output of any git operation 
(such

as git log --show-signature, git show, etc) - these are all processed
and do not reveal what is actually going on.  I want a copy of the
actual file containing the signature.  If the signature is embedded 
in

the commit then I want the file in the objects directory tree that
contains the commit.  They're just compressed text files (though it 
is

a bit of a pain to decompress them).


Not nessecarily. Given that :

 a file with a given content has a fixed SHA
A tree is just a list of these SHA's , and that in turn is 
referenced
by SHA, so if 2 commits have identical file content, their 'tree' 
sha

will be the same ( in theory ).

So that means, if you perform a rebase, assuming the filesystem 
looks

the same as it did before the rebase, it will have the same SHA1 for
the tree, regardless of the process of how it got to be that way.


The filesystem WON'T look the same after a rebase.

quick example (operations done in this order):

file in commit A in master:
1
2
3
4
5

file in commit B in a branch off of master:
1
2a
3
4
5

file in commit C in master:
1
2
3
4a
5

if you merge master into the branch you'll end up with a new commit D
whose parent is B that looks like:
1
2a
3
4a
5

if instead you rebase master into the branch you'll end up with a new
commit D whose parent is C that looks like:
1
2a
3
4a
5

The history for the branch will no longer contain B.  If there were 
14

commits B1..14 you'd end up with 14 commits D1.14 that each contain
the line 4a.  None of them would use the same trees as B1..14, and
they can't use the same signatures as a result, even if only the tree
was signed.   As far as the new history was concerned, line 4a was
there before the branch was started.

At least, that is my understanding of rebasing.  Again, I'm a bit of 
a

git novice, but I never really grokked git until I saw a presentation
that didn't start with commands, but instead started out with just
walking through the actual structure of the repository.  Once you 
grok

the COW model I find it all clicks into place.

Rich


--
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, 
Gatchina, Russia

Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexx...@gmail.com
mailto:ale...@gentoo.org
mailto:ale...@omrb.pnpi.spb.ru



[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Duncan
William Hubbs posted on Thu, 31 May 2012 15:57:14 -0500 as excerpted:

 On Thu, May 31, 2012 at 08:26:58PM +, Duncan wrote:
 William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:
 
 I don't know what's going to happen to all the overlays with the main
 tree switch to git, but won't that break various overlay first
 policies, say for the kde overlay?
 
 Of course, if all the official overlays are converted to git branches
 of the main tree... but won't they still require rebasing as they've
 already been pushed?  (This assumes your workaround idea doesn't work. 
 If it does, great!)
 
 Overlays aren't really part of this discussion; those are independent
 trees which we have no control over, so commiting changes from overlays
 to the main tree is the responsibility of the overlay maintainers.

But it seems to me that overlays are the primary use case for commits to 
public trees other than gentoo first.  Otherwise, the whole rebase-vs-
merge problem goes away, because the first public commit is to the gentoo 
tree.  But especially with overlays (like kde) that have an overlay-
first, test, then gentoo-tree, policy, that public overlay tree (which is 
already in git) is part of the process.  Commits MUST go thru the overlay 
to get to the tree, and that overlay is public, so constant rebasing is a 
definite no-no.

Which unless your workaround idea works, pretty much leaves us with merge-
commits as a necessity.  (Which of course, as Ciaran pointed out, are 
part of the point of git, such that running git without merge-commits 
defeats part of the purpose of the whole exercise.)

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Nicolas Sebrecht
The 01/06/12, Duncan wrote:

 But it seems to me that overlays are the primary use case for commits to 
 public trees other than gentoo first.  Otherwise, the whole rebase-vs-
 merge problem goes away, because the first public commit is to the gentoo 
 tree.  But especially with overlays (like kde) that have an overlay-
 first, test, then gentoo-tree, policy, that public overlay tree (which is 
 already in git) is part of the process.  Commits MUST go thru the overlay 
 to get to the tree, and that overlay is public, so constant rebasing is a 
 definite no-no.

The purpose of overlays is to have ebuilds maintained outside of the
official Gentoo portage. Importing a ebuild from an overlay whether it
uses Git or not means importing the ebuild(s). In the Git context, it
means the Gentoo maintainer has to make an import commit the same way
it would be done to start a project with somthing like:

  cp 'files to be commited' .
  git add .
  git commit -m 'initial import'

So, like explained before your concern is clearly out of the current
discussion. Importing commit history from Overlays is not supported and
will probably never be. Gentoo doesn't forces (and doesn't want to)
overlays maintainers to use Git.

-- 
Nicolas Sebrecht



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 2 June 2012 00:42, Rich Freeman ri...@gentoo.org wrote:
 On Fri, Jun 1, 2012 at 7:23 AM, Kent Fredric kentfred...@gmail.com wrote:

 Nope, at least not as far as I can tell, and I just implemented commit
 signature verification _

 I've been trying to find an example of a signed commit, but can't find
 one anywhere, so it is hard to tell what it is doing without going
 through the git source carefully.  If you have an example of a signed
 commit feel free to send it to me.

 Note that I am NOT interested in the output of any git operation (such
 as git log --show-signature, git show, etc) - these are all processed
 and do not reveal what is actually going on.  I want a copy of the
 actual file containing the signature.  If the signature is embedded in
 the commit then I want the file in the objects directory tree that
 contains the commit.  They're just compressed text files (though it is
 a bit of a pain to decompress them).

git cat-file -p $sha is as close as you can get to commit objects
without needing to write your own decompressing wrapper.  But it gives
the same results.

Here is a sample signed git commit ( sans compression ) :

https://gist.github.com/2852363

Line 5 to 21 are where the GPG signature is stored. ( Git uses the
leading space on all the lines of the GPG sig to indicate that its
part of the signature, line 22 does not lead with a space, so that
line is after the GPG signature.  Line 22 also is empty, which ends
the header section of the commit, and there starts the comment
section.

If you don't believe me that git cat-file -p $sha is reliable, here
is a simple script that has no git internals in it , its just Zlib
decompression : https://gist.github.com/2852411

perl /tmp/deflate.pl .git/objects/66/50c09ad7b2a62e28476e639654443015ac5945

And that re-emits exactly the same thing : https://gist.github.com/2852363

And if you want to do that yourself, here's that object in compressed
form base64 encoded https://gist.github.com/2852436


 The filesystem WON'T look the same after a rebase.

 quick example (operations done in this order):

 file in commit A in master:
 1
 2
 3
 4
 5

 file in commit B in a branch off of master:
 1
 2a
 3
 4
 5

 file in commit C in master:
 1
 2
 3
 4a
 5

 if you merge master into the branch you'll end up with a new commit D
 whose parent is B that looks like:
 1
 2a
 3
 4a
 5

 if instead you rebase master into the branch you'll end up with a new
 commit D whose parent is C that looks like:
 1
 2a
 3
 4a
 5

 The history for the branch will no longer contain B.  If there were 14
 commits B1..14 you'd end up with 14 commits D1.14 that each contain
 the line 4a.  None of them would use the same trees as B1..14, and
 they can't use the same signatures as a result, even if only the tree
 was signed.   As far as the new history was concerned, line 4a was
 there before the branch was started.

It all depends really on how you do the rebase, if the changes in the
rebase shadow all the competing changes , or the changes are shadowed
out before the rebase, the rebase will be the same afterwards, just
with a different history.

ie:

  root - a - b - c - d( backout of c )

If you had started a branch off b, and then rebased that branch on
top of d, the tree would be in the exact same state, but the history
would have changed.

And likewise, if the change was :

  root - a - b - c - d - e

And you started a branch at b, with commits m, n, o, p , changing the
same lines c, d and e changed,
and when you rebased, you opted to let the changes in m, n, and o
replace the changes in c, d and e,
( which has the same effect as if you had deleted c , d and e, except
they're still there in the history ), when you move the branch from b
to e, the last commit of that branch (p) will still be the same.

The rebased commits m, n, and o will be different to what they were
when they were on b, because now they're not only making changes vs
b, but they're also replacing code added in c, d, and e.

Sure, this only happens in certain circumstances, and usually when it
happens, its because you want it to happen , but it does happen, and
its quite useful sometimes.

( essentially, x + y + z == x can be true, as long as z = -y  )


 At least, that is my understanding of rebasing.  Again, I'm a bit of a
 git novice, but I never really grokked git until I saw a presentation
 that didn't start with commands, but instead started out with just
 walking through the actual structure of the repository.  Once you grok
 the COW model I find it all clicks into place.

Yeah, chapter 9 of the Pro Git book is where I recommend everyone with
even a sliver of comp-sci knowledge to start.
http://www.git-scm.com/book/en/Git-Internals

Once you see its basically a bunch of text files, that refer to each
other via sha1 , and are stored in files named after their sha1, (
after a bit of compression ), all that graph theory just clicks and
you wonder how they managed to make the UI so complicated =p




-- 

Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Andreas K. Huettel
 The purpose of overlays is to have ebuilds maintained outside of the
 official Gentoo portage. Importing a ebuild from an overlay whether it
 uses Git or not means importing the ebuild(s). In the Git context, it
 means the Gentoo maintainer has to make an import commit the same way
 it would be done to start a project with somthing like:
 
   cp 'files to be commited' .
   git add .
   git commit -m 'initial import'
 
 So, like explained before your concern is clearly out of the current
 discussion. Importing commit history from Overlays is not supported and
 will probably never be. 

Which does not mean that it's not desirable. 

The KDE ebuilds are mainly evolving inside the KDE overlay, where KDE betas 
and live ebuilds live. Being able (in the future) to see the history of an 
ebuild before it was imported into the main tree would certainly help in some 
cases.


-- 
Andreas K. Huettel
Gentoo Linux developer
kde, sci, arm, tex, printing


signature.asc
Description: This is a digitally signed message part.


Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Andreas K. Huettel
 git cat-file -p $sha is as close as you can get to commit objects
 without needing to write your own decompressing wrapper.  But it gives
 the same results.

Now, does the signed data also contain the parent sha?

If yes, our discussion about rebasing is moot, because a rebase will in every 
case destroy previous signatures.

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, sci, arm, tex, printing


signature.asc
Description: This is a digitally signed message part.


Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 2 June 2012 03:12, Andreas K. Huettel dilfri...@gentoo.org wrote:
 git cat-file -p $sha is as close as you can get to commit objects
 without needing to write your own decompressing wrapper.  But it gives
 the same results.

 Now, does the signed data also contain the parent sha?

 If yes, our discussion about rebasing is moot, because a rebase will in every
 case destroy previous signatures.


Yes. Which basically means, you *cannot* have both

a) rebase only merges
and
b) every commit must be signed

as policies.

At very best, I think either
a) a future git might support signed rebases ( ie: replacing existing
signatures with new signatures in the name of the person performing
the rebase )
or
b) somebody could write a wrapper that provides signed-rebase support
until git get around to implementing it natively.

and even then, you're going to lose original signing info ( Though,
thats no worse than the signer of the manifest file changing every
sign )


-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 11:12 AM, Andreas K. Huettel
dilfri...@gentoo.org wrote:
 Now, does the signed data also contain the parent sha?


So, I was working on a lengthy email which now would be fairly
repetitive of what Kent posted.

Suffice it to say I managed to rip out a commit from the kde overlay,
deflate it, and compared that the signature:

-BEGIN PGP SIGNATURE-
 Version: GnuPG v2.0.18 (GNU/Linux)

 iQEcBAABCgAGBQJPx+mcAAoJEO+t9ga+3I3aqLoH/0OrRA1+NPRHGfbbLoQrqMwl
 sB+2It2Pb9LfPjEme+lrQu5WgFY4j7k0qd2ZYdnXM7JdQjsqmpfAMloHh5JN4TAS
 4vk8+u2GJCYgzL/SY5XlPl2l8dT91PhQJSN0yVt4Q9TsoN3nzVpFBjACJCy9R6j2
 HrXvz/g3+MqY/9VesV8IiVgvQUTVgCdh8zBJ2rVyWAVH0bErsn518aiwEyfzNOxA
 1qJxxgGJLMpXp+nI8rnmhqTAAKiNA+byAKAsTEl3LS7OvQZ51aOCwa4A2GLOn2ef
 5JmuYQG5/FsS0RfXrqk72PiStTBWa3TakHYrgNXOXlslIR5AIB2tYnXqZcdEqYQ=
 =fucY
 -END PGP SIGNATURE-

does in fact verify for the payload:
--start--
tree 7d7f97cded40158d0f580ca6fbe97398d5c867f8
parent 14d7d9cb2219f64c7a715d8da0bbe48a32c9dad8
author Johannes Huber j...@gentoo.org 1338501525 +0200
committer Johannes Huber j...@gentoo.org 1338501525 +0200

[kde-base/kdelibs] Sync with live.

(Portage version: 2.2.0_alpha108/git/Linux i686, unsigned Manifest commit)
--end--

Dump those into a text file and run gpg for yourself...  The full
commit contains the gpg signature in a field as already posted by
Kent.

And while I appreciate the performance boost and space savings
provided by all the compression/packing/etc, I've learned to almost
hate those features with a passion this morning...  Getting a cloned
repo unpacked, and the commit decompressed was a bit pita.  The other
issue is that the header in the commit file is stripped before it is
signed, the actual start of the commit is commit 830tree...

Rich



Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Andreas K. Huettel
 On 2 June 2012 03:12, Andreas K. Huettel dilfri...@gentoo.org wrote:
  git cat-file -p $sha is as close as you can get to commit objects
  without needing to write your own decompressing wrapper.  But it gives
  the same results.
  
  Now, does the signed data also contain the parent sha?
  
  If yes, our discussion about rebasing is moot, because a rebase will in
  every case destroy previous signatures.
 
 Yes. Which basically means, you *cannot* have both
 
 a) rebase only merges
 and
 b) every commit must be signed
 
 as policies.
 

I would say that this is a very strong argument in favour of allowing merge 
commits.

-- 
Andreas K. Huettel
Gentoo Linux developer
kde, sci, arm, tex, printing


signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Michał Górny
On Sat, 2 Jun 2012 03:25:43 +1200
Kent Fredric kentfred...@gmail.com wrote:

 On 2 June 2012 03:12, Andreas K. Huettel dilfri...@gentoo.org wrote:
  git cat-file -p $sha is as close as you can get to commit objects
  without needing to write your own decompressing wrapper.  But it
  gives the same results.
 
  Now, does the signed data also contain the parent sha?
 
  If yes, our discussion about rebasing is moot, because a rebase
  will in every case destroy previous signatures.
 
 
 Yes. Which basically means, you *cannot* have both
 
 a) rebase only merges
 and
 b) every commit must be signed
 
 as policies.
 
 At very best, I think either
 a) a future git might support signed rebases ( ie: replacing existing
 signatures with new signatures in the name of the person performing
 the rebase )
 or
 b) somebody could write a wrapper that provides signed-rebase support
 until git get around to implementing it natively.
 
 and even then, you're going to lose original signing info ( Though,
 thats no worse than the signer of the manifest file changing every
 sign )

Yes, it's no worse so we're practically considering implementing a very
complex mechanism for no real benefit.

As I see it, as good would be only requiring some kind of 'top-level'
commits to be signed. For example, if one does merge a branch
to the tree, a signed merge commit should already be good enough for
us. Not sure if git is able to do that, however...

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 9:25 AM, Nicolas Sebrecht nsebre...@piing.fr wrote:
 So, like explained before your concern is clearly out of the current
 discussion. Importing commit history from Overlays is not supported and
 will probably never be. Gentoo doesn't forces (and doesn't want to)
 overlays maintainers to use Git.

I'm not sure that git even supports this, unless the overlay is a
complete clone of the entire gentoo-x86.git repository.

I think the way git operations work is by finding common parents in
the history of two heads, and then moving forward from there.  If you
have two completely different repositories then they never will have a
common parent.

Plus, even if it did work, to rebase the overlay on the gentoo-x86
repository you'd have to import the full gentoo-x86 tree into it.
Then you'd have to push EVERYTHING in your overlay into gentoo-x86.  I
don't think you can just push individual files from one tree to
another.

From what I've seen the various methods out there which do involve
moving only part of one branch into another basically involve a LOT of
history manipulation or are really no different than just copying the
files into the branch and adding them, losing all history in the
process.

I'm not sure how important all that history preservation actually is -
the overlay would still possess it.  If we did want to preserve it
then the only practical option I see is to have the overlay start out
as a clone of gentoo-x86 and keep around all the non-overlay packages,
which then means that anybody using the overlay will get all those old
gentoo-x86 packages as part of their portage tree.  Plus you still
have the rebase+gpg-signatures=fail issue.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread William Hubbs
On Fri, Jun 01, 2012 at 12:57:10PM +, Duncan wrote:
 William Hubbs posted on Thu, 31 May 2012 15:57:14 -0500 as excerpted:
  Overlays aren't really part of this discussion; those are independent
  trees which we have no control over, so commiting changes from overlays
  to the main tree is the responsibility of the overlay maintainers.
 
 But it seems to me that overlays are the primary use case for commits to 
 public trees other than gentoo first.  Otherwise, the whole rebase-vs-
 merge problem goes away, because the first public commit is to the gentoo 
 tree.  But especially with overlays (like kde) that have an overlay-
 first, test, then gentoo-tree, policy, that public overlay tree (which is 
 already in git) is part of the process.  Commits MUST go thru the overlay 
 to get to the tree, and that overlay is public, so constant rebasing is a 
 definite no-no.
 
 Which unless your workaround idea works, pretty much leaves us with merge-
 commits as a necessity.  (Which of course, as Ciaran pointed out, are 
 part of the point of git, such that running git without merge-commits 
 defeats part of the purpose of the whole exercise.)

Overlays are completely separate repositories. There is nothing stopping
an overlay from using git right now even if the main tree isn't using
git. They just work in their git repositories until they are ready to
commit something to the main tree, then they move the changes to the
main tree.

What the main tree on git would give them is the ability to create a
branch from the main tree for their changes, but that would not be pushed
to the main repository.

All they would have to do when they are ready for their code to be
merged back into the main repository is make sure that they are creating
a fast-forward merge so that there is no merge commit on the master
branch.

William



pgpmzpgQTqKJR.pgp
Description: PGP signature


Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 11:36 AM, Andreas K. Huettel
dilfri...@gentoo.org wrote:
 On 2 June 2012 03:12, Andreas K. Huettel dilfri...@gentoo.org wrote:
 Yes. Which basically means, you *cannot* have both

 a) rebase only merges
 and
 b) every commit must be signed

 as policies.


 I would say that this is a very strong argument in favour of allowing merge
 commits.

One advantage of merge commits with signatures is that the history
really does reflect who signed what.

Proxy maintainer signs a bunch of ebuilds.  I merge them in.  The
commits show that the proxy maintainer signed a bunch of work done
against an old tree, and I signed a bunch of merge diffs that
basically synced them up to the new tree.

However, this is missing another issue.  What is the value of
preserving all those original signatures in the first place?  I'd
think that they'd mainly be used as some kind of web-of-trust.  Well,
would such a web-of-trust include proxy maintainers in the first
place?

If you want the tree to be traceable to Gentoo devs, then rewriting
the signatures is probably a good thing.

However, Kent did point out the rebase function doesn't actually apply
new signatures to the new old commits anyway, so you'd end up with
unsigned commits in the history.

git-rebase is just a shell script, that ultimately just calls
git-commit as far as I can see, which means that implementing
re-signing is just a matter of adding the appropriate parameters, or
use default configuration (assuming it doesn't already do this).

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 2 June 2012 03:39, Rich Freeman ri...@gentoo.org wrote:
 On Fri, Jun 1, 2012 at 9:25 AM, Nicolas Sebrecht nsebre...@piing.fr wrote:
 So, like explained before your concern is clearly out of the current
 discussion. Importing commit history from Overlays is not supported and
 will probably never be. Gentoo doesn't forces (and doesn't want to)
 overlays maintainers to use Git.

 I'm not sure that git even supports this, unless the overlay is a
 complete clone of the entire gentoo-x86.git repository.

 I think the way git operations work is by finding common parents in
 the history of two heads, and then moving forward from there.  If you
 have two completely different repositories then they never will have a
 common parent.

You can however merge dissimilar histories with no common parents if
you know what you're doing. It does warn you, but it still lets you do
it.

 Plus, even if it did work, to rebase the overlay on the gentoo-x86
 repository you'd have to import the full gentoo-x86 tree into it.
 Then you'd have to push EVERYTHING in your overlay into gentoo-x86.  I
 don't think you can just push individual files from one tree to
 another.

Yeah, selectively pulling in files with histories however is hard,
I've occasionally been fussed enough to create temporary copies of
branches, and then iterate over their history so the history is
reduced to changes only for a small selection of files, but its
largely lots of work, and I can't remember how to do it every time I
go to do it -_-

Its reasonably straight forward to cherry-pick commits in though.

 From what I've seen the various methods out there which do involve
 moving only part of one branch into another basically involve a LOT of
 history manipulation or are really no different than just copying the
 files into the branch and adding them, losing all history in the
 process.

 I'm not sure how important all that history preservation actually is -
 the overlay would still possess it.  If we did want to preserve it
 then the only practical option I see is to have the overlay start out
 as a clone of gentoo-x86 and keep around all the non-overlay packages,
 which then means that anybody using the overlay will get all those old
 gentoo-x86 packages as part of their portage tree.  Plus you still
 have the rebase+gpg-signatures=fail issue.

 Rich


Myself, I'd be inclined to do something like this:

1. Have an integration branch on gx86
2. Select a package or packages to migrate to gx86 from an overlay
3. create a branch in the overlay that only contains the history with
regard to those packages.
4. copy the branch to a local checkout of gx86, and then rebase it on
top of the integration branch
5. then delete it from the overlay.

But the exact part of Create a branch that contains only the history
of those packages is the part I need to get down to an art .

-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Dirkjan Ochtman
On Fri, Jun 1, 2012 at 5:53 PM, Rich Freeman ri...@gentoo.org wrote:
 If you want the tree to be traceable to Gentoo devs, then rewriting
 the signatures is probably a good thing.

I'd say that signing the merge commit is good enough. It says the
Gentoo dev who merged it has reviewed the changes and can be held
responsible for them. One could even say that he mediates a
web-of-trust to the more casual contributor who signed the original
csets.

Cheers,

Dirkjan



Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 2 June 2012 03:53, Rich Freeman ri...@gentoo.org wrote:

 git-rebase is just a shell script, that ultimately just calls
 git-commit as far as I can see, which means that implementing
 re-signing is just a matter of adding the appropriate parameters, or
 use default configuration (assuming it doesn't already do this).

 Rich


Oh that makes it slightly easier, I didn't realise it was just a blob
of bash doing all that O.o

grep -nR 'git commit' git-rebase*
git-rebase--interactive.sh:152: warn   git commit --amend
git-rebase--interactive.sh:441: git commit --amend --no-post-rewrite || 
{
git-rebase--interactive.sh:484: do_with_author output git commit
--no-verify -F $squash_msg ||
git-rebase--interactive.sh:491: do_with_author git 
commit
--no-verify -F $fixup_msg ||
git-rebase--interactive.sh:496: do_with_author git 
commit --no-verify -e ||
git-rebase--interactive.sh:699:  git commit --amend
git-rebase--interactive.sh:703:  git commit
git-rebase--interactive.sh:723: do_with_author git commit --no-verify
-F $msg -e || {
git-rebase--merge.sh:30:if ! git commit --no-verify -C $cmt
git-rebase--merge.sh:32:echo Commit failed, please do 
not call
\git commit\


Throwing '-S' in there liberally aught to do the trick.



-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread W. Trevor King
On Sat, Jun 02, 2012 at 03:56:43AM +1200, Kent Fredric wrote:
 You can however merge dissimilar histories with no common parents if
 you know what you're doing. It does warn you, but it still lets you do
 it.
 
 …
 
 Yeah, selectively pulling in files with histories however is hard,
 I've occasionally been fussed enough to create temporary copies of
 branches, and then iterate over their history so the history is
 reduced to changes only for a small selection of files, but its
 largely lots of work, and I can't remember how to do it every time I
 go to do it -_-

I have to look it up every time I need it too, but the procedure
involves `git filter-branch` [1,2], and actually is pretty easy to
wrap your head around, even though the paricular commands are not so
easy to remember.

Trevor

[1]: http://stackoverflow.com/questions/7375528/
[2]: 
http://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Robin H. Johnson
On Fri, Jun 01, 2012 at 10:45:48AM -0500, William Hubbs wrote:
 Overlays are completely separate repositories. There is nothing stopping
 an overlay from using git right now even if the main tree isn't using
 git. They just work in their git repositories until they are ready to
 commit something to the main tree, then they move the changes to the
 main tree.
What about overlay repositories that elect to be a branch of the main
tree via git?

Do we call that forbidden usage?

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Robin H. Johnson
On Fri, Jun 01, 2012 at 11:53:52AM -0400, Rich Freeman wrote:
 However, Kent did point out the rebase function doesn't actually apply
 new signatures to the new old commits anyway, so you'd end up with
 unsigned commits in the history.
Only in your local history. The push to the central repo would only send
the commits in the active chain to your branch HEAD.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
On 2 June 2012 04:33, Robin H. Johnson robb...@gentoo.org wrote:
 On Fri, Jun 01, 2012 at 10:45:48AM -0500, William Hubbs wrote:
 Overlays are completely separate repositories. There is nothing stopping
 an overlay from using git right now even if the main tree isn't using
 git. They just work in their git repositories until they are ready to
 commit something to the main tree, then they move the changes to the
 main tree.
 What about overlay repositories that elect to be a branch of the main
 tree via git?

 Do we call that forbidden usage?

You can't practically use any overlay foolish enough to publish these
repositories for end user consumption.

Its just a silly idea. There's no problem with having overlays cloned
into a branch as a step towards it hitting mainline, but overlays
being distributed to users as main tree branches is just a silly idea.

 Mostly, because end-users will still have ::gentoo via rsync, and the
load of cloning a git repo of ::gentoo will be too much for the
average user, doing that just to get an overlay is exhaustively
execessive vs the current mechanism we have for overlays, and it comes
at a penalty at being not as good as overlays in that you can't easily
have 1 of them.

-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Kent Fredric
 Only in your local history. The push to the central repo would only send
 the commits in the active chain to your branch HEAD.

Any commits that are rebased, and then replicated somewhere after that
rebase, will be stripped of their signatures by the rebase process.



-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: Re: Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Robin H. Johnson
On Sat, Jun 02, 2012 at 04:56:33AM +1200, Kent Fredric wrote:
  Only in your local history. The push to the central repo would only send
  the commits in the active chain to your branch HEAD.
 Any commits that are rebased, and then replicated somewhere after that
 rebase, will be stripped of their signatures by the rebase process.
I'm saying the old signatures will not be pushed out for verification,
only the new signatures will.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 12:33 PM, Robin H. Johnson robb...@gentoo.org wrote:
 What about overlay repositories that elect to be a branch of the main
 tree via git?

 Do we call that forbidden usage?

I think that branches off of the main tree are mainly going to be
useful for more eclass/profile/etc-related work.  Work on a package or
small group of packages will almost always go better in a completely
independent overlay.  If you try to do that kind of work in a branch
then if you create an rsync tree from that branch it will contain all
the other packages that you aren't working on, and they'll get stale
very quickly.  Anybody using that as an overlay will get a bunch of
old ebuilds for who-knows-what in their tree.

Now, branches in the main tree are going to be really useful for stuff
like package moves, new virtuals, eclass api changes, or other messy
changes that have big tree-wide impacts.  You can stage the change and
fix the 300 impacted ebuilds in a branch, get lots of people to test
them, and then merge those in with a single transaction, pulling in
updates from master all the while.  That is more about portage tree
maintenance than package maintenance per-se.

All that said, having the tree in git is still a big help to proxy
maintainers and others even with all these issues.  I've worked as an
outside contributor to other projects and it is a lot easier with git.
 I can easily work in my own PM, rebase against their master, and then
easily submit a nice clean diff as a patch, even without doing any
pushing at all.  I don't have to have push rights to anything official
to be more efficient in my contributions.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread William Hubbs
On Fri, Jun 01, 2012 at 04:33:35PM +, Robin H. Johnson wrote:
 On Fri, Jun 01, 2012 at 10:45:48AM -0500, William Hubbs wrote:
  Overlays are completely separate repositories. There is nothing stopping
  an overlay from using git right now even if the main tree isn't using
  git. They just work in their git repositories until they are ready to
  commit something to the main tree, then they move the changes to the
  main tree.
 What about overlay repositories that elect to be a branch of the main
 tree via git?
 
 Do we call that forbidden usage?

No, it just would mean that they have to know how to rebase their
changes on master before they commit.

They would clone the main tree to their overlay location, then make a
kde branch in their location. That branch would be where they did their
work and they would keep master in sync with the main tree (upstream) by
running git pull.

To merge the overlay changes into the master, a developer would use a
git remote to add the kde overlay to his tree, add a kde branch that
tracked the remote kde branch, rebase that on current master, merge it
into master to create a fast-forward merge then push.

I think that probably sounds more complicated than it is, so let me know
if you need clarification.

William



pgpglcTCfr3Mr.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Michael Weber
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/31/2012 02:04 PM, Aaron W. Swenson wrote:
 The 6 hours it takes to clone the repo.
afaik it's 6 hours to transform the whole cvs history into a git repo.

Cloning the repo [1] takes 200seconds on 8cores (it's 2GB of data and
22 minutes of 3.4GHz cpus).

[1] git clone git://git-exp.overlays.gentoo.org/exp/gentoo-x86.git
[2] http://lore.xmw.de/gentoo/gentoo-exp/notes

Michael
- --
Gentoo Dev
http://xmw.de/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAk/JKucACgkQknrdDGLu8JAIxAEAhLZ4Tk6rXy1qAUbDDHS4UYiL
gGUPj5PKUKSS1YJp6hkA/R9aEqjSNr/8iZ2novNFGjvbJ5CtDkvI+PsvMTUNDoDo
=3t+7
-END PGP SIGNATURE-



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread James Cloos
 WH == William Hubbs willi...@gentoo.org writes:

WH My big complaint about merge commits is if you do a git show hash on
WH a merge commit, you get nothing,

With current git and proper merge logs you get useful info.

   The headers contain the hashes, so you can get the list of
   commits pulled by that merge.

   The signed tag log is show.

   And merge conflicts also are shown.

Based on the hashes in the Merge: header, you can use git log to see the
individual commits or git diff to see the whole picture at once.

Linus’ current tip is a good example:

 cd .../linux
 git show 1193755ac632

So, Gentoo shouldn't prohibit merges.  Instead, it should demand that
all merges be of signed tags.

The plan includes signed commits anyway, so signed tags for pulls will be
fully supported by any version of git which might be used.

-JimC
-- 
James Cloos cl...@jhcloos.com OpenPGP: 1024D/ED7DAEA6



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-06-01 Thread Rich Freeman
On Fri, Jun 1, 2012 at 4:49 PM, Michael Weber x...@gentoo.org wrote:

 Cloning the repo [1] takes 200seconds on 8cores (it's 2GB of data and
 22 minutes of 3.4GHz cpus).


As others have pointed out, probably the best way to bootstrap this is
to offer tarballs of a shallow repository and a full repository.
Perhaps we'd offer the latter as a torrent.  The shallow repository
should be light on the CPU too.

This would be a lot easier on the server than having everybody and
their uncle doing a full 2GB clone.

Devs could then do a pull to get the latest and greatest, and that
would only transfer the delta.

I imagine our mirror network can handle the bandwidth compared to what
we're already doing with distfiles.  Worst case we could take a
one-time hit and use S3 or whatever to do the distribution (they even
support bittorrent to cut down on the bill).

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Aaron W. Swenson
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote:
 On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson
 robb...@gentoo.org wrote:
 No, the last mock conversion is still live and updating fairly
 often: 
 http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary

 
 Since you seem to know most about this project, can you give a
 short summary on what *you* think are hard blockers for the
 migration?
 
 Cheers,
 
 Dirkjan
 

The 6 hours it takes to clone the repo.

- - Aaron
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAk/HXjoACgkQVxOqA9G7/aB2DwD+JqdcMJDIfu30Oht8rB6H/pMY
Wr1RjhujZvVGIDQG55QA/jPlsWh3c8El7HjxzhjoCNIPkV1Vj3b7VZVc/D6y6oq9
=FxGY
-END PGP SIGNATURE-



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Peter Stuge
Aaron W. Swenson wrote:
  what *you* think are hard blockers for the migration?
 
 The 6 hours it takes to clone the repo.

Maybe clone on server and distribute the initial repo as tarball.


//Peter



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Dirkjan Ochtman
On Thu, May 31, 2012 at 2:04 PM, Aaron W. Swenson titanof...@gentoo.org wrote:
 The 6 hours it takes to clone the repo.

IIRC someone already proposed that the packed repo could be offered
via normal download (or even BitTorrent).

Cheers,

Dirkjan



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Robin H. Johnson
On Wed, May 30, 2012 at 10:31:06PM +0200, Dirkjan Ochtman wrote:
 On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson robb...@gentoo.org wrote:
  No, the last mock conversion is still live and updating fairly often:
  http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary
 Since you seem to know most about this project, can you give a short
 summary on what *you* think are hard blockers for the migration?
There are only two items from my perspective.
1. 
Discussion on merge policy. Originally I thought we would disallow merge
commits, so that we would get a cleaner history. However, it turns out that if
the repo ends up being pushed to different places with slightly different
histories, merges are absolutely going to be required to prevent somebody from
having to rebase at least one of their sets of commits that are already pushed.

2. 
Git-SVN breakage. Why does this matter you're wondering?
We need the newer Git for the commit signing, but it comes with a
price, the git-svn binary has some major failures with SVN 1.7.
Git since 1.7.8 has been broken this way.

You can see them best with the testsuite. I haven't keyworded those ebuilds for
git at all, because git-svn in some of my tests ended up being destructive of
the repository - it actually lost data.

The error sometimes turns up like this.

Initialized empty Git repository in 
/dev/shm/portage/dev-vcs/git-/work/git-/t/t d.t9155/git_project/.git/
svn: E235000: In file 'subversion/libsvn_subr/dirent_uri.c' line 2291: 
assertion failed (svn_uri_is_canonical(url, pool))

This is seemingly due to a change in SVN 1.7 that is stricter about all paths
(both URIs and to files/directories) being properly escaped.

Upstream Git has made no progress on it in more than 6 months.
Both ferringb and I put several weeks of work into it without succeeding.

If you want to reproduce it:
- Upgrade your SVN to 1.7
- Build  run the git testsuite
- Most of the t91xx tests will fail, because the working dir is 'trash 
directory'.
- If you patch the working dir to be 'trash_directory', you'll be left with
  failures in at least: t9100 t9118 t9120
  Because those are the tests that have whitespace or other characters that
  need escaping.

The last version of my patch is files/git-1.7.8-git-svn-1.7-canonical-path.patch
but it caused more problems than it fixed.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Robin H. Johnson
On Thu, May 31, 2012 at 08:04:10AM -0400, Aaron W. Swenson wrote:
 On 05/30/2012 04:31 PM, Dirkjan Ochtman wrote:
  On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson
  robb...@gentoo.org wrote:
  No, the last mock conversion is still live and updating fairly
  often: 
  http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary
 The 6 hours it takes to clone the repo.
To directly clone that repo yes.

Which you should NEVER be doing in reality.

The final conversion repo we have already stated will only be 40MB, with
no history. History will be available separately to graft on if you need
it, just like the Linux kernel did with historical data.

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Rich Freeman
On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote:
 1.
 Discussion on merge policy. Originally I thought we would disallow merge
 commits, so that we would get a cleaner history. However, it turns out that if
 the repo ends up being pushed to different places with slightly different
 histories, merges are absolutely going to be required to prevent somebody from
 having to rebase at least one of their sets of commits that are already 
 pushed.

Not sure I'm following, but I will be the first to admit that I'm a
git novice.  Would this be aided by a convention, like only committing
to master on the gentoo official repository, and any on-the-side work
on places like github/etc stays in branches?  Those repositories would
just keep getting fed commits on master from the official repository.


 2.
 Git-SVN breakage. Why does this matter you're wondering?
 We need the newer Git for the commit signing, but it comes with a
 price, the git-svn binary has some major failures with SVN 1.7.
 Git since 1.7.8 has been broken this way.

To clarify - these won't be issues for gentoo per se, but there is a
sense that we can't stabilize the latest git because it will break it
for people using git-svn on non-gentoo work?

I think the general conclusion was that we would not be supporting any
mixed git+cvs/svn/etc workflows for gentoo itself.

If that is the case, what is our sense of how important this feature
even is to gentoo developers?  They're the only ones who really have
to have the latest git to commit to the official tree.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Robin H. Johnson
On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
 On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote:
  1.
  Discussion on merge policy. Originally I thought we would disallow merge
  commits, so that we would get a cleaner history. However, it turns out that 
  if
  the repo ends up being pushed to different places with slightly different
  histories, merges are absolutely going to be required to prevent somebody 
  from
  having to rebase at least one of their sets of commits that are already 
  pushed.
 Not sure I'm following, but I will be the first to admit that I'm a
 git novice.  Would this be aided by a convention, like only committing
 to master on the gentoo official repository, and any on-the-side work
 on places like github/etc stays in branches?  Those repositories would
 just keep getting fed commits on master from the official repository.
Ok, let me try and reword my statement.

- You have a commit, that you want to put into the Gentoo tree.
- You have already pushed it to your github, signed
- It needs to be merged/rebased so that it applies on the Gentoo tree.
- If you force it to be a rebase so it applies on the tip, then you may
  have changed the history of your github tree, and broken any further
  forks.
- If you permit a merge instead, nobody gets broken.

  2.
  Git-SVN breakage. Why does this matter you're wondering?
  We need the newer Git for the commit signing, but it comes with a
  price, the git-svn binary has some major failures with SVN 1.7.
  Git since 1.7.8 has been broken this way.
 To clarify - these won't be issues for gentoo per se, but there is a
 sense that we can't stabilize the latest git because it will break it
 for people using git-svn on non-gentoo work?
As the Git maintainer, I will not keyword it for anybody until I know
it's not going to lose/corrupt data, regardless of what they are using
it for.

I don't think there are many SVN repos left in Gentoo that haven't
converted to using Git directly, so it's probably not a problem from
that side.

 If that is the case, what is our sense of how important this feature
 even is to gentoo developers?  They're the only ones who really have
 to have the latest git to commit to the official tree.
You'd be excluding me entirely, I need to use git-svn for other work
projects, and emerging between two different versions of git would be
very annoying (I switch constantly between the sides of work as they
overlap).

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread William Hubbs
On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
 On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson robb...@gentoo.org wrote:
  1.
  Discussion on merge policy. Originally I thought we would disallow merge
  commits, so that we would get a cleaner history. However, it turns out that 
  if
  the repo ends up being pushed to different places with slightly different
  histories, merges are absolutely going to be required to prevent somebody 
  from
  having to rebase at least one of their sets of commits that are already 
  pushed.
 
 Not sure I'm following, but I will be the first to admit that I'm a
 git novice.  Would this be aided by a convention, like only committing
 to master on the gentoo official repository, and any on-the-side work
 on places like github/etc stays in branches?  Those repositories would
 just keep getting fed commits on master from the official repository.

 Iagree with this; I think we should ban merge commits on master. That
 would force everyone to rebase their work on current master before they
 commit to master which would make the history clean.

William


pgpA0ksxOrWmJ.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Ciaran McCreesh
On Thu, 31 May 2012 14:18:04 -0500
William Hubbs willi...@gentoo.org wrote:
  Not sure I'm following, but I will be the first to admit that I'm a
  git novice.  Would this be aided by a convention, like only
  committing to master on the gentoo official repository, and any
  on-the-side work on places like github/etc stays in branches?
  Those repositories would just keep getting fed commits on master
  from the official repository.
 
  Iagree with this; I think we should ban merge commits on master. That
  would force everyone to rebase their work on current master before
 they commit to master which would make the history clean.

So what's the point of switching to git if you want to ban the main
reason git exists?

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Rich Freeman
On Thu, May 31, 2012 at 3:13 PM, Robin H. Johnson robb...@gentoo.org wrote:
 - You have a commit, that you want to put into the Gentoo tree.
 - You have already pushed it to your github, signed
 - It needs to be merged/rebased so that it applies on the Gentoo tree.
 - If you force it to be a rebase so it applies on the tip, then you may
  have changed the history of your github tree, and broken any further
  forks.
 - If you permit a merge instead, nobody gets broken.

Maybe the best compromise is to tell people that if you push to
master on other repositories, you get to deal with the mess.  If we
try to keep side overlays/etc working on branches and not on master
then there will be no history to rewrite, as the merge will be rebased
when it hits the official master, and from there it will get pulled by
other repositories.

We can perhaps allow merge commits on other branches, where the
continuity of history is less important.

Does that make sense?

 You'd be excluding me entirely, I need to use git-svn for other work
 projects, and emerging between two different versions of git would be
 very annoying (I switch constantly between the sides of work as they
 overlap).

I'm a big proponent of letting the people doing the work scratch their
own itches first!  However, this does make us dependent on upstream -
is there any sense of when they'll be ready, or what their own
priority is for this issue.  If this is becoming a deprecated feature
then I'm not sure we can tie our future to it.

I wasn't sure if any of the existing git-svn bugs pertained to this
issue.  Either way we should add this as a blocker to the git
migration tracker.

I think that even if we made a big push it would still take us a month
or two to be ready with docs/infra/etc, and that might be optimistic.
So, this might not be rate-limiting if upstream is actively working on
it.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Michał Górny
On Thu, 31 May 2012 14:18:04 -0500
William Hubbs willi...@gentoo.org wrote:

 On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
  On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson
  robb...@gentoo.org wrote:
   1.
   Discussion on merge policy. Originally I thought we would
   disallow merge commits, so that we would get a cleaner history.
   However, it turns out that if the repo ends up being pushed to
   different places with slightly different histories, merges are
   absolutely going to be required to prevent somebody from having
   to rebase at least one of their sets of commits that are already
   pushed.
  
  Not sure I'm following, but I will be the first to admit that I'm a
  git novice.  Would this be aided by a convention, like only
  committing to master on the gentoo official repository, and any
  on-the-side work on places like github/etc stays in branches?
  Those repositories would just keep getting fed commits on master
  from the official repository.
 
  Iagree with this; I think we should ban merge commits on master. That
  would force everyone to rebase their work on current master before
 they commit to master which would make the history clean.

What would git signing work with rebased commits? Would all of them
have to be signed once again?

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Alexey Shvetsov

Michał Górny писал 2012-05-31 23:33:

On Thu, 31 May 2012 14:18:04 -0500
William Hubbs willi...@gentoo.org wrote:


On Thu, May 31, 2012 at 01:48:29PM -0400, Rich Freeman wrote:
 On Thu, May 31, 2012 at 12:49 PM, Robin H. Johnson
 robb...@gentoo.org wrote:
  1.
  Discussion on merge policy. Originally I thought we would
  disallow merge commits, so that we would get a cleaner history.
  However, it turns out that if the repo ends up being pushed to
  different places with slightly different histories, merges are
  absolutely going to be required to prevent somebody from having
  to rebase at least one of their sets of commits that are already
  pushed.

 Not sure I'm following, but I will be the first to admit that I'm 
a

 git novice.  Would this be aided by a convention, like only
 committing to master on the gentoo official repository, and any
 on-the-side work on places like github/etc stays in branches?
 Those repositories would just keep getting fed commits on master
 from the official repository.

 Iagree with this; I think we should ban merge commits on master. 
That

 would force everyone to rebase their work on current master before
they commit to master which would make the history clean.


What would git signing work with rebased commits? Would all of them
have to be signed once again?


Commits itsels still will be signed
--
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, NRC Kurchatov Institute, 
Gatchina, Russia

Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexx...@gmail.com
mailto:ale...@gentoo.org
mailto:ale...@omrb.pnpi.spb.ru



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread William Hubbs
On Thu, May 31, 2012 at 07:13:42PM +, Robin H. Johnson wrote:
 - You have a commit, that you want to put into the Gentoo tree.
 - You have already pushed it to your github, signed

If I have a github tree, that would probably be because I didn't have
push access to the official tree, so signing the commit probably
isn'tgoing to matter; I would expect that a gentoo dev who has push
access to the tree would sign the commit when it is put into the gentoo
tree.

 - It needs to be merged/rebased so that it applies on the Gentoo tree.
 - If you force it to be a rebase so it applies on the tip, then you may
   have changed the history of your github tree, and broken any further
   forks.
 - If you permit a merge instead, nobody gets broken.

If you do this:

git rebase master mybranch
git checkout master
git merge mybranch -- this is a fast-forward merge
git pull --rebase
git push

I think that covers this concern doesn't it?

   2.
   Git-SVN breakage. Why does this matter you're wondering?
   We need the newer Git for the commit signing, but it comes with a
   price, the git-svn binary has some major failures with SVN 1.7.
   Git since 1.7.8 has been broken this way.
  To clarify - these won't be issues for gentoo per se, but there is a
  sense that we can't stabilize the latest git because it will break it
  for people using git-svn on non-gentoo work?
 As the Git maintainer, I will not keyword it for anybody until I know
 it's not going to lose/corrupt data, regardless of what they are using
 it for.

Why not keyword it and use package.use.mask for the git-svn flag?

William


pgpwkzFYxv1GM.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Rich Freeman
On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote:
 What would git signing work with rebased commits? Would all of them
 have to be signed once again?


The whole point of rebasing is to throw away history (which is either
good or bad based on your perspective).

So, if 14 devs spend 3 years and 2000 commits working on something in
a branch, and I commit it to master using a rebase, then all you'll
see in the master history is that rich0 committed 20k lines of code to
master on May 31st, and that would be signed by me.

I think that rebasing before merging is a pretty typical workflow
anyway - when you submit a patch to Linus, he doesn't really care that
you spent six months tweaking it - he just is getting a big blob of
code that either works or doesn't.  Does all that sub-history really
matter?  You could always push the branch to the repository if you
wanted to keep it on the side.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Michał Górny
On Thu, 31 May 2012 15:58:43 -0400
Rich Freeman ri...@gentoo.org wrote:

 On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org
 wrote:
  What would git signing work with rebased commits? Would all of them
  have to be signed once again?
 
 
 The whole point of rebasing is to throw away history (which is either
 good or bad based on your perspective).
 
 So, if 14 devs spend 3 years and 2000 commits working on something in
 a branch, and I commit it to master using a rebase, then all you'll
 see in the master history is that rich0 committed 20k lines of code to
 master on May 31st, and that would be signed by me.
 
 I think that rebasing before merging is a pretty typical workflow
 anyway - when you submit a patch to Linus, he doesn't really care that
 you spent six months tweaking it - he just is getting a big blob of
 code that either works or doesn't.  Does all that sub-history really
 matter?  You could always push the branch to the repository if you
 wanted to keep it on the side.

That sounds like a pretty poor workflow to me. If I tweak something for
3 years, I'm likely to have a larger set of logically organized commits.
I'm not saying it's unlikely I'm going to rebase fixes for obvious
mistakes but putting everything onto one blob just makes the changes
harder to read and less maintainable.

For example, if I first create a basic function and then add additional
options to it, I'm likely to keep those as separate commits. If one of
the changes was a really bad idea, I'd like to revert the appropriate
commit rather than removing all traces of it by hand and mistakenly
introducing even worse breakage.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Duncan
William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:

 On Thu, May 31, 2012 at 07:13:42PM +, Robin H. Johnson wrote:
 - You have a commit, that you want to put into the Gentoo tree.
 - You have already pushed it to your github, signed
 
 If I have a github tree, that would probably be because I didn't have
 push access to the official tree, so signing the commit probably
 isn'tgoing to matter; I would expect that a gentoo dev who has push
 access to the tree would sign the commit when it is put into the gentoo
 tree.

I don't know what's going to happen to all the overlays with the main 
tree switch to git, but won't that break various overlay first 
policies, say for the kde overlay?

Of course, if all the official overlays are converted to git branches of 
the main tree... but won't they still require rebasing as they've already 
been pushed?  (This assumes your workaround idea doesn't work.  If it 
does, great!)

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Michael Orlitzky
On 05/31/12 16:09, Michał Górny wrote:
 On Thu, 31 May 2012 15:58:43 -0400
 Rich Freeman ri...@gentoo.org wrote:
 
 On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org
 wrote:
 What would git signing work with rebased commits? Would all of them
 have to be signed once again?


 The whole point of rebasing is to throw away history (which is either
 good or bad based on your perspective).

 So, if 14 devs spend 3 years and 2000 commits working on something in
 a branch, and I commit it to master using a rebase, then all you'll
 see in the master history is that rich0 committed 20k lines of code to
 master on May 31st, and that would be signed by me.

 I think that rebasing before merging is a pretty typical workflow
 anyway - when you submit a patch to Linus, he doesn't really care that
 you spent six months tweaking it - he just is getting a big blob of
 code that either works or doesn't.  Does all that sub-history really
 matter?  You could always push the branch to the repository if you
 wanted to keep it on the side.
 
 That sounds like a pretty poor workflow to me. If I tweak something for
 3 years, I'm likely to have a larger set of logically organized commits.
 I'm not saying it's unlikely I'm going to rebase fixes for obvious
 mistakes but putting everything onto one blob just makes the changes
 harder to read and less maintainable.
 
 For example, if I first create a basic function and then add additional
 options to it, I'm likely to keep those as separate commits. If one of
 the changes was a really bad idea, I'd like to revert the appropriate
 commit rather than removing all traces of it by hand and mistakenly
 introducing even worse breakage.
 

That isn't what rebasing does, unless you do an interactive rebase and
decide to squash the commits.

The usual use case for a rebase is just to avoid the ugly merge commit.
For example,

  1. I clone your portage tree
  2. I start making commits locally
  3. You add a new package
  4. I try to push my changes to you

Step 4 doesn't work because your repo has changed. I can either,

  a) pull from you again (merge)
  b) pull with a rebase

And then I should be able to push to you, assuming there were no conflicts.

The merge option works fine but generates an ugly merge commit. The
rebase takes our two histories and combines them into a nice linear
history. In this case, a rebase would (essentially) take all of my
commits and stick them on top of your new package commit. Afterwards,
it looks like there's a nice linear history: you added a package, and
then I did a bunch of other stuff. All of the commits are there.

Since that history is linear and it looks like just a bunch of stuff on
top of your existing tree, I can push it to you without problems.

The only downside to the rebase is that it modifies my local history.
So, if somebody cloned *my* repo in the middle of that, they could get
screwed up.



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread William Hubbs
On Thu, May 31, 2012 at 08:26:58PM +, Duncan wrote:
 William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:
 
 I don't know what's going to happen to all the overlays with the main 
 tree switch to git, but won't that break various overlay first 
 policies, say for the kde overlay?
 
 Of course, if all the official overlays are converted to git branches of 
 the main tree... but won't they still require rebasing as they've already 
 been pushed?  (This assumes your workaround idea doesn't work.  If it 
 does, great!)

Overlays aren't really part of this discussion; those are independent
trees which we have no control over, so commiting changes from overlays
to the main tree is the responsibility of the overlay maintainers.

William



pgpnUoF15VbKy.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread William Hubbs
On Thu, May 31, 2012 at 03:58:43PM -0400, Rich Freeman wrote:
 On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote:
  What would git signing work with rebased commits? Would all of them
  have to be signed once again?
 
 
 The whole point of rebasing is to throw away history (which is either
 good or bad based on your perspective).
 
 So, if 14 devs spend 3 years and 2000 commits working on something in
 a branch, and I commit it to master using a rebase, then all you'll
 see in the master history is that rich0 committed 20k lines of code to
 master on May 31st, and that would be signed by me.

You don't commit to master with a rebase,; it is always a merge. The
type of merge is what controls what you see in the logs.

If you rebase your branch on master, merge to master then run git pull
--rebase then push, you will get a fast forward merge, which shows the
individual commits.

If you don't include the rebasing, you get another type of merge which
just shows up in the logs as one commit afaik.

William



pgpGov9QxG7kv.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Kent Fredric
On 1 June 2012 07:52, Alexey Shvetsov ale...@gentoo.org wrote:

 What would git signing work with rebased commits? Would all of them
 have to be signed once again?


 Commits itsels still will be signed


Do you know how git does this? Do you have experience/information you
can cite as to that this works?

Commit signing seems poorly documented at present, and I've been
looking at the git internals, and it would *APPEAR* that the content
that is signed is the blob of text you normally get when you

   git cat-file -p $SHA1

And indeed, if you  git cat-file -p $SHA1  file, extract the
SIGNATURE part into its own file (removing the leading spaces), and
remove the gnupg section from the commit headers,   gpg --verify
$sigfile $file   # tells me I have a good signature.

Just I haven't worked out what happens when the SHA1 of the 'parent'
header changes, which *will* change if the rebase is anything other
than a fast-forward.

If that SHA1 changes, the gpg signature will surely fail?


-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Kent Fredric
On 1 June 2012 07:58, Rich Freeman ri...@gentoo.org wrote:
 On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org wrote:
 What would git signing work with rebased commits? Would all of them
 have to be signed once again?


 The whole point of rebasing is to throw away history (which is either
 good or bad based on your perspective).

 So, if 14 devs spend 3 years and 2000 commits working on something in
 a branch, and I commit it to master using a rebase, then all you'll
 see in the master history is that rich0 committed 20k lines of code to
 master on May 31st, and that would be signed by me.

 I think that rebasing before merging is a pretty typical workflow
 anyway - when you submit a patch to Linus, he doesn't really care that
 you spent six months tweaking it - he just is getting a big blob of
 code that either works or doesn't.  Does all that sub-history really
 matter?  You could always push the branch to the repository if you
 wanted to keep it on the side.

 Rich


I think you're conflating rebasing and squashing commits.

You should rebase a long commit sequence and squash pointless fixup
commits, and to make the commit sequence logical and ordered, possibly
divided by logical changes that one may wish to later revert. ( That
way, backing out a problem is simply reversing that commit and
applying the reversal patch )

You should not rebase for the purpose of squashing an entire history
of changes into a single scattered commit.

Rebasing is more to make the history itself linear and non-complex,
as when walking backwards through history, there being 2 parallel
histories that generated a merged commit can be confusing for humans,
so eliminating the parallel histories is one of the primary purposes I
advocate use of rebase for.

Squashed commits are a handy feature of rebase, but I wouldn't want to
see an entire overlay squashed into the main tree as a single squashed
commit.

Another bad reason for squashed commits: if you're getting rid of the
Changes files, you'll have no history on anything if you just group
long histories into a single commit. None.



-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread William Hubbs
On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote:
 On Thu, 31 May 2012 14:18:04 -0500
 William Hubbs willi...@gentoo.org wrote:
   Not sure I'm following, but I will be the first to admit that I'm a
   git novice.  Would this be aided by a convention, like only
   committing to master on the gentoo official repository, and any
   on-the-side work on places like github/etc stays in branches?
   Those repositories would just keep getting fed commits on master
   from the official repository.
  
   Iagree with this; I think we should ban merge commits on master. That
   would force everyone to rebase their work on current master before
  they commit to master which would make the history clean.
 
 So what's the point of switching to git if you want to ban the main
 reason git exists?

To clarify: we should only allow fast-forward merges on master.

My big complaint about merge commits is if you do a git show hash on
a merge commit, you get nothing, so there is no way to see what actually
changed in that commit.

William



pgpLhNVh63xgN.pgp
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Kent Fredric
On 1 June 2012 08:26, Duncan 1i5t5.dun...@cox.net wrote:
 William Hubbs posted on Thu, 31 May 2012 14:54:50 -0500 as excerpted:
 Of course, if all the official overlays are converted to git branches of
 the main tree... but won't they still require rebasing as they've already
 been pushed?  (This assumes your workaround idea doesn't work.  If it
 does, great!)


End users will still want to work with overlays that are not merged
with the main tree, not merely git branches.

Its foreseeable that there will be git branches that /track/ overlays
and exist as an integration pipeline for content from the overlays
joining core gentoo, but end users will not want to use that.

For the simple reason of course, that as soon as you want 1 overlay,
portage's way of doing it with separate repositories is far more
effective.

You don't want each user to have to maintain an octopus merge between
all the branches they want to have commits from ;)

-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Rich Freeman
On Thu, May 31, 2012 at 5:52 PM, Kent Fredric kentfred...@gmail.com wrote:
 Just I haven't worked out what happens when the SHA1 of the 'parent'
 header changes, which *will* change if the rebase is anything other
 than a fast-forward.

 If that SHA1 changes, the gpg signature will surely fail?

Rebasing doesn't modify past commits - it creates new ones and the
past ones are no longer in the history of the current head.  So, it
doesn't break the old signatures so much as discard them.  You would
need to create new signatures in their place, presumably from whoever
performed the rebase.

I'm trying to glean what I can from the little info out there about
how the new commit signatures work, but I don't think that you can use
signatures to convey authorship if later authors are going to rebase
the branch.  The situation is not unlike what we have now with
manifests.

As far as I can tell if you want to do work outside of master, and
then get those commits into master but preserve all the past
signatures in the history of master, then you'd need to do a merge
commit, so that all the deltas to do the merge are in a separate
commit which is then signed by the person doing the merge.

Rich



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Michał Górny
On Thu, 31 May 2012 17:04:30 -0500
William Hubbs willi...@gentoo.org wrote:

 On Thu, May 31, 2012 at 08:23:31PM +0100, Ciaran McCreesh wrote:
  On Thu, 31 May 2012 14:18:04 -0500
  William Hubbs willi...@gentoo.org wrote:
Not sure I'm following, but I will be the first to admit that
I'm a git novice.  Would this be aided by a convention, like
only committing to master on the gentoo official repository,
and any on-the-side work on places like github/etc stays in
branches? Those repositories would just keep getting fed
commits on master from the official repository.
   
Iagree with this; I think we should ban merge commits on master.
   That would force everyone to rebase their work on current master
   before they commit to master which would make the history clean.
  
  So what's the point of switching to git if you want to ban the main
  reason git exists?
 
 To clarify: we should only allow fast-forward merges on master.
 
 My big complaint about merge commits is if you do a git show hash on
 a merge commit, you get nothing, so there is no way to see what
 actually changed in that commit.

Or you use a graphical tool which shows the whole merge history and you
see the exact changes happening rather than some blob with 'do foo, do
bar, and some baz too'.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Michał Górny
On Thu, 31 May 2012 16:27:48 -0400
Michael Orlitzky mich...@orlitzky.com wrote:

 On 05/31/12 16:09, Michał Górny wrote:
  On Thu, 31 May 2012 15:58:43 -0400
  Rich Freeman ri...@gentoo.org wrote:
  
  On Thu, May 31, 2012 at 3:33 PM, Michał Górny mgo...@gentoo.org
  wrote:
  What would git signing work with rebased commits? Would all of
  them have to be signed once again?
 
 
  The whole point of rebasing is to throw away history (which is
  either good or bad based on your perspective).
 
  So, if 14 devs spend 3 years and 2000 commits working on something
  in a branch, and I commit it to master using a rebase, then all
  you'll see in the master history is that rich0 committed 20k lines
  of code to master on May 31st, and that would be signed by me.
 
  I think that rebasing before merging is a pretty typical workflow
  anyway - when you submit a patch to Linus, he doesn't really care
  that you spent six months tweaking it - he just is getting a big
  blob of code that either works or doesn't.  Does all that
  sub-history really matter?  You could always push the branch to
  the repository if you wanted to keep it on the side.
  
  That sounds like a pretty poor workflow to me. If I tweak something
  for 3 years, I'm likely to have a larger set of logically organized
  commits. I'm not saying it's unlikely I'm going to rebase fixes for
  obvious mistakes but putting everything onto one blob just makes
  the changes harder to read and less maintainable.
  
  For example, if I first create a basic function and then add
  additional options to it, I'm likely to keep those as separate
  commits. If one of the changes was a really bad idea, I'd like to
  revert the appropriate commit rather than removing all traces of it
  by hand and mistakenly introducing even worse breakage.
  
 
 That isn't what rebasing does, unless you do an interactive rebase and
 decide to squash the commits.

Yes, it isn't but such kind of work flow was presented in the message I
replied to.

 The usual use case for a rebase is just to avoid the ugly merge
 commit.

Which devs should simply do whenever they use the scenario you
mentioned. I agree we could block 'auto-merges' when pushing to
a modified repo but not disallow a valid merges from devs who know what
they're doing.

-- 
Best regards,
Michał Górny


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-31 Thread Kent Fredric
On 1 June 2012 14:49, Rich Freeman ri...@gentoo.org wrote:
 On Thu, May 31, 2012 at 5:52 PM, Kent Fredric kentfred...@gmail.com wrote:
 Just I haven't worked out what happens when the SHA1 of the 'parent'
 header changes, which *will* change if the rebase is anything other
 than a fast-forward.

 If that SHA1 changes, the gpg signature will surely fail?

 Rebasing doesn't modify past commits - it creates new ones and the
 past ones are no longer in the history of the current head.  So, it
 doesn't break the old signatures so much as discard them.  You would
 need to create new signatures in their place, presumably from whoever
 performed the rebase.


Hmm, thats annoying. Almost makes me wish it was the trees that were
signed, not the commits.

Although, I probably could brew up a prototype resigning tool ( based
on Git::PurePerl ,... when they accept and publish my changes ) , just
would be problematic because simply the act of signing a past commit
means the SHA1 of the commit itself is different, so all successive
commits after a re-signed commit will change and also need to be
rebased and re-signed.

-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Duncan
Rich Freeman posted on Tue, 29 May 2012 21:55:04 -0400 as excerpted:

 So, what is the big issue?  Is there something not being tracked, or is
 one of those items a lot harder than it looks?

I'd suggest that it's like openrc stabilization.  The biggest problem 
with it is that it's a BIG job, and the simplest solution is to simply 
have someone commit to it, with full council *priority* backing, and push 
and push until it's done.

There *is* one huge don't-do-it-that-way lesson to take from the openrc 
stabilization, tho:  Get the documentation in place BEFORE throwing the 
switch.  I'm still not sure what happened with openrc.  The 
documentation bug was a blocker... until it was the last one and then 
suddenly it went live without proper upgrade documentation, or that's the 
way it seemed from here, anyway.  The fact that we had essentially no doc-
project to work on the documentation was unfortunately a problem, the one 
guy still trying to hang on in docs so backlogged and burnt out that it 
was about hopeless, action time stretching toward infinity, but swift 
coming back on board dramatically improved that situation so at least it 
shouldn't be an infinity blocker, now.

But really what that means is that whoever ends up taking charge of that 
final push, needs to be prepared to learn gentoo's docs CSS definitions 
and do it themself, if it comes to that.

Meanwhile, the positive takeaway from the openrc stabilization is that 
someone suitably determined, along with council backing and everyone else 
rowing the same way where their little part of gentoo comes into contact 
with the job at hand, goes a long way!

Of course, there's a much larger infra component to the git migration, so 
either having that someone being an infra person, or at least having 
someone from infra have the time and be willing to work closely with 
them, is going to be critical.  But again, given a council *priority*, 
let's move on it! decision, I'd at least /hope/ that's not a blocker.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Dirkjan Ochtman
On Wed, May 30, 2012 at 11:38 AM, Duncan 1i5t5.dun...@cox.net wrote:
 Of course, there's a much larger infra component to the git migration, so
 either having that someone being an infra person, or at least having
 someone from infra have the time and be willing to work closely with
 them, is going to be critical.  But again, given a council *priority*,
 let's move on it! decision, I'd at least /hope/ that's not a blocker.

Yeah... this is why I was asking about access to infra to test the
conversion; so far, I haven't had any replies, though.

Cheers,

Dirkjan



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Rich Freeman
On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman d...@gentoo.org wrote:
 Yeah... this is why I was asking about access to infra to test the
 conversion; so far, I haven't had any replies, though.


A mock conversion would probably help with creating
procedures/docs/etc as well.  It is nice to say that we're just going
to use git but I think everybody has a slightly different picture of
how that is going to work.

If we could set up an official unofficial portage tree in git based
on a one-time migration (maybe refreshing it from time to time) that
could be a sandbox used to work things out, and it would then be
replaced with the official tree.  When the official migration comes
along we'd already be experts in doing it.

All we need to do is execute the migration, and just not point the
rsync generation process at it.  Maybe it won't be perfectly right at
first, and that would basically be the point of doing it.  Devs could
update tools to work against it, and the docs could be written
alongside.

Rich



[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Duncan
Rich Freeman posted on Wed, 30 May 2012 07:32:49 -0400 as excerpted:

 On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman d...@gentoo.org wrote:
 Yeah... this is why I was asking about access to infra to test the
 conversion; so far, I haven't had any replies, though.


 A mock conversion would probably help with creating procedures/docs/etc
 as well.  It is nice to say that we're just going to use git but I
 think everybody has a slightly different picture of how that is going to
 work.

I /believe/ such mock conversions had been done, based on previous 
threads here, tho NOT on the SCM-migration list, which would certainly 
have the greater detail.  AFAIK the previous sticking points were all 
those still-open bugs, not the general conversion.  But the bugs, other 
than documentation, either seem mostly fixed or not so important, any 
more.

Of course, those previous trial runs are probably similarly dated, now, 
so a new one's probably in order with the new perspective on those bug 
priorities, etc, but at least getting the input of someone that was 
involved with them should speed progress over some already covered 
ground, in any case.

(I've considered following scmm myself, but there's apparently some sort 
of issue between gmane, pan, and lists commonly crossposted between, that 
has already killed project for me, header-fetch does nothing, and scmm 
would almost certainly be similarly dead to me.)

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Tobias Klausmann
Hi! 

On Wed, 30 May 2012, Rich Freeman wrote:
 On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman d...@gentoo.org wrote:
  Yeah... this is why I was asking about access to infra to test the
  conversion; so far, I haven't had any replies, though.
 
 A mock conversion would probably help with creating
 procedures/docs/etc as well.  It is nice to say that we're just going
 to use git but I think everybody has a slightly different picture of
 how that is going to work.

I recommend having a smallish set of willing alpha/beta testers for
this. This usually helps with some of the near-edge cases. You'll
still find a thousand other bugs once things go live for
everybody. Still, it turns a million into a thousand. It also
gives you slightly more realistic load test.

 If we could set up an official unofficial portage tree in git based
 on a one-time migration (maybe refreshing it from time to time) that
 could be a sandbox used to work things out, and it would then be
 replaced with the official tree.  When the official migration comes
 along we'd already be experts in doing it.

This is a good idea that goes nicely with what I wrote above.

 All we need to do is execute the migration, and just not point the
 rsync generation process at it.  Maybe it won't be perfectly right at
 first, and that would basically be the point of doing it.  Devs could
 update tools to work against it, and the docs could be written
 alongside.

The scientist in me wonders how big the dent in productivity
will be, actually. After all, there's going to be a lot of people
that will hammer the new setup just because of the New! Shiny!
appeal.

Regards,
Tobias



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Aaron W. Swenson
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/30/2012 12:53 PM, Tobias Klausmann wrote:
 Hi!
 
 On Wed, 30 May 2012, Rich Freeman wrote:
 On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman d...@gentoo.org
 wrote:
 Yeah... this is why I was asking about access to infra to test
 the conversion; so far, I haven't had any replies, though.
 
 A mock conversion would probably help with creating 
 procedures/docs/etc as well.  It is nice to say that we're just
 going to use git but I think everybody has a slightly different
 picture of how that is going to work.
 
 I recommend having a smallish set of willing alpha/beta testers
 for this. This usually helps with some of the near-edge cases.
 You'll still find a thousand other bugs once things go live for 
 everybody. Still, it turns a million into a thousand. It also gives
 you slightly more realistic load test.  Regards, Tobias
 

As one that's relatively new to Git, I'm a perfect candidate for
testing. If we go this way, I volunteer.

- - Aaron
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAk/GUZkACgkQVxOqA9G7/aDxogEAjEsS6k0M9TejvCAoYISvuk1u
odo5ecLsyshwvwRlk3sA+wUeVAfGrafgH1gKpSrEedk1OT5x7HfsMquK0owq0/Np
=mzDU
-END PGP SIGNATURE-



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Justin
On 30.05.2012 18:58, Aaron W. Swenson wrote:
 On 05/30/2012 12:53 PM, Tobias Klausmann wrote:
 Hi!
 
 On Wed, 30 May 2012, Rich Freeman wrote:
 On Wed, May 30, 2012 at 6:16 AM, Dirkjan Ochtman d...@gentoo.org
 wrote:
 Yeah... this is why I was asking about access to infra to test
 the conversion; so far, I haven't had any replies, though.

 A mock conversion would probably help with creating 
 procedures/docs/etc as well.  It is nice to say that we're just
 going to use git but I think everybody has a slightly different
 picture of how that is going to work.
 
 I recommend having a smallish set of willing alpha/beta testers
 for this. This usually helps with some of the near-edge cases.
 You'll still find a thousand other bugs once things go live for 
 everybody. Still, it turns a million into a thousand. It also gives
 you slightly more realistic load test.  Regards, Tobias
 
 
 As one that's relatively new to Git, I'm a perfect candidate for
 testing. If we go this way, I volunteer.
 
 - Aaron
 

As I am a hater of CVS and my current checkout is repeatability spiting
weird errors, I would join as a tester here.

justin



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Robin H. Johnson
On Wed, May 30, 2012 at 01:06:45PM +, Duncan wrote:
 Of course, those previous trial runs are probably similarly dated, now, 
 so a new one's probably in order with the new perspective on those bug 
 priorities, etc, but at least getting the input of someone that was 
 involved with them should speed progress over some already covered 
 ground, in any case.
No, the last mock conversion is still live and updating fairly often:
http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary

-- 
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee  Infrastructure Lead
E-Mail : robb...@gentoo.org
GnuPG FP   : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-30 Thread Dirkjan Ochtman
On Wed, May 30, 2012 at 8:33 PM, Robin H. Johnson robb...@gentoo.org wrote:
 No, the last mock conversion is still live and updating fairly often:
 http://git-exp.overlays.gentoo.org/gitweb/?p=exp/gentoo-x86.git;a=summary

Since you seem to know most about this project, can you give a short
summary on what *you* think are hard blockers for the migration?

Cheers,

Dirkjan



[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Duncan
Dan Douglas posted on Thu, 24 May 2012 01:04:48 -0500 as excerpted:

 On Thursday, May 24, 2012 07:56:58 AM Michał Górny wrote:
 On Wed, 23 May 2012 16:14:53 -0500
 
 Dan Douglas orm...@gmail.com wrote:
  If not I will be leaving Gentoo for Funtoo in the near future, though
  there are disadvantages to doing this I don't look forward to dealing
  with.
 
 Most of us will probably be doing that :P.
 
 Eh sorry that wasn't meant to be antagonistic. I'll still have Gentoo
 boxen to deal with. I just need to be able to use git on the tree (even
 without the full history is perfectly fine) to ease the difficulty of
 local overlay management. Glad to hear that will be possible, or at
 least somewhat easier.

FWIW, I as a user would sure like a git-based tree.  Doing git whatchanged 
searches on individual files and being able to track my last checkout and 
roll back to it, or to a point between it and current HEAD, are extremely 
useful.  I haven't thought of it much until now, but I think maintaining 
overlays as simple branches would be great, as well.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Dan Douglas
On Thursday, May 24, 2012 06:33:53 AM Duncan wrote:
 Dan Douglas posted on Thu, 24 May 2012 01:04:48 -0500 as excerpted:
  On Thursday, May 24, 2012 07:56:58 AM Michał Górny wrote:
  On Wed, 23 May 2012 16:14:53 -0500
 
  Dan Douglas orm...@gmail.com wrote:
   If not I will be leaving Gentoo for Funtoo in the near future, though
   there are disadvantages to doing this I don't look forward to dealing
   with.
 
  Most of us will probably be doing that :P.
 
  Eh sorry that wasn't meant to be antagonistic. I'll still have Gentoo
  boxen to deal with. I just need to be able to use git on the tree (even
  without the full history is perfectly fine) to ease the difficulty of
  local overlay management. Glad to hear that will be possible, or at
  least somewhat easier.

 FWIW, I as a user would sure like a git-based tree.  Doing git whatchanged
 searches on individual files and being able to track my last checkout and
 roll back to it, or to a point between it and current HEAD, are extremely
 useful.  I haven't thought of it much until now, but I think maintaining
 overlays as simple branches would be great, as well.

I don't think doing a branch of the entire tree is a good idea (well
maybe...). I was thinking more along the lines of subtree merges into a local
overlay, or perhaps submodules. To do that currently (I think) would require
taking the rsync tree and putting that into a repo, and trying to keep it
synchronized. Plus in the process you lose all correspondance with upstream
commits so that logs and diffs become meaningless.
--
Dan Douglas

signature.asc
Description: This is a digitally signed message part.


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Vítor Brandão
2012/5/24 Dan Douglas orm...@gmail.com

 On Thursday, May 24, 2012 06:33:53 AM Duncan wrote:
  Dan Douglas posted on Thu, 24 May 2012 01:04:48 -0500 as excerpted:
   On Thursday, May 24, 2012 07:56:58 AM Michał Górny wrote:
   On Wed, 23 May 2012 16:14:53 -0500
  
   Dan Douglas orm...@gmail.com wrote:
If not I will be leaving Gentoo for Funtoo in the near future,
 though
there are disadvantages to doing this I don't look forward to
 dealing
with.
  
   Most of us will probably be doing that :P.
  
   Eh sorry that wasn't meant to be antagonistic. I'll still have Gentoo
   boxen to deal with. I just need to be able to use git on the tree (even
   without the full history is perfectly fine) to ease the difficulty of
   local overlay management. Glad to hear that will be possible, or at
   least somewhat easier.
 
  FWIW, I as a user would sure like a git-based tree.  Doing git
 whatchanged
  searches on individual files and being able to track my last checkout and
  roll back to it, or to a point between it and current HEAD, are extremely
  useful.  I haven't thought of it much until now, but I think maintaining
  overlays as simple branches would be great, as well.

 I don't think doing a branch of the entire tree is a good idea (well
 maybe...). I was thinking more along the lines of subtree merges into a
 local
 overlay, or perhaps submodules. To do that currently (I think) would
 require
 taking the rsync tree and putting that into a repo, and trying to keep it
 synchronized. Plus in the process you lose all correspondance with upstream
 commits so that logs and diffs become meaningless.
 --
 Dan Douglas


git++

-- 
Vítor Brandão  (noisebleed)


[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Duncan
Dan Douglas posted on Thu, 24 May 2012 01:51:28 -0500 as excerpted:

 I don't think doing a branch of the entire tree is a good idea (well
 maybe...). I was thinking more along the lines of subtree merges into a
 local overlay, or perhaps submodules. To do that currently (I think)
 would require taking the rsync tree and putting that into a repo, and
 trying to keep it synchronized. Plus in the process you lose all
 correspondance with upstream commits so that logs and diffs become
 meaningless.

The thing about git branches is that they cost almost nothing.  If the 
whole main tree is a single git tree, and you already have that 
available, maintaining a branch consisting of what we now call an overlay 
is trivial, compared to simply maintaining the separate files, as is 
normally done now.

In that regard, git is nothing like for instance svn, where branches come 
at a much higher cost, as does merging between them.  (CVS... I don't 
actually know enough about to make an informed comparison.

It'd be a real shame not to expose the read-only git tree to the users 
who want it.  Git was /designed/ to be distributed in that manner.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman




Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Dirkjan Ochtman
On Thu, May 24, 2012 at 1:43 PM, Duncan 1i5t5.dun...@cox.net wrote:
 In that regard, git is nothing like for instance svn, where branches come
 at a much higher cost, as does merging between them.

That's wrong. SVN branches are just about as cheap as git branches,
although merges used to be much more painful. I'm not sure how good
merging in recent SVN is.

Let's please stay a little on-topic? The migration will get there much
faster if we don't succumb to feature creep.

Cheers,

Dirkjan



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Ciaran McCreesh
On Thu, 24 May 2012 14:05:50 +0200
Dirkjan Ochtman d...@gentoo.org wrote:
 On Thu, May 24, 2012 at 1:43 PM, Duncan 1i5t5.dun...@cox.net wrote:
  In that regard, git is nothing like for instance svn, where
  branches come at a much higher cost, as does merging between them.
 
 That's wrong. SVN branches are just about as cheap as git branches,

[citation needed]

-- 
Ciaran McCreesh


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Kent Fredric
On 25 May 2012 00:05, Dirkjan Ochtman d...@gentoo.org wrote:
 On Thu, May 24, 2012 at 1:43 PM, Duncan 1i5t5.dun...@cox.net wrote:
 In that regard, git is nothing like for instance svn, where branches come
 at a much higher cost, as does merging between them.

 That's wrong. SVN branches are just about as cheap as git branches,
 although merges used to be much more painful. I'm not sure how good
 merging in recent SVN is.

Cheapness ... maybe in binary disk utilization ( need an actual
comparison here I think ), but in cognitive overheads, I'd argue git's
branching system is definitely cheaper.  Going from Git back to SVN,
the mentality of copy a directory and you have a new branch!!! seems
a bit crazy.

And switching between branches in-place at a fixed disk location is
definitely cheaper ( mentally at least ) than SVN.

I hope I never have to use svn switch again :/
-- 
Kent

perl -e  print substr( \edrgmaM  SPA NOcomil.ic\\@tfrken\, \$_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );

http://kent-fredric.fox.geek.nz



Re: [gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-24 Thread Michael Weber
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On 05/24/2012 03:37 PM, Kent Fredric wrote:

Kent, this is of topic, stop it.

- -- 
- --
Gentoo Dev
http://xmw.de/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAk++PO8ACgkQknrdDGLu8JC2cAD/WknV6DJEzYEsuXJD0TuDU99I
arDTkrBHNXydYVdaxGoBAJmmVm3o7YWlMAvFBz2Eu4ma/VXEHdqocFwl0NK+FEAh
=Esu3
-END PGP SIGNATURE-



[gentoo-dev] Re: Portage Git migration - clean cut or git-cvsserver

2012-05-23 Thread Michael Palimaka

On 2012-05-23 22:42, Michael Weber wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hi,

i've looked at the blockers of [TRACKER] portage migration to git
[1] and want to discuss testing git-cvsserver [2].

There are two proposed scenarios how to migrate the developers write
access to the portage tree.

Clean cut turns of cvs access on a given and announced timestamp,
rsync-generation/updates is suspended (no input -  no changes), some
magic scripts prepare the git repo (according to [3], some hours
duration) and we all checkout the tree (might be some funny massive load).

testing git-cvsserver proses Clean cut with the additional ability
to continue using cvs update/commit, - in best case - on the old
checkout w/o alteration on the developers side.

Clean cut forces us to clean up out dirty checkouts (I have some
added directories, added ebuilds i hesitated to `repoman commit`).
Plus we have to alter all our hot-wired portage mangling scripts from
cvs'ish to git'ish (I use my read/write checkout as portage tree (cvs
checkout + egencache for checkout) and have an automated google-chrome
bump script). But this can be accomplished on a per developer basis,
and slackers don't stall the process.

testing git-cvsserver forces us all to test these cvs'ish scripts
and behaviours against a git-cvsserver and report.
We all know that this test-runs will never happen, stalling this bug
till infinity.
Plus infra/subset of devs marshalling the migration get stuck
between fixing git issues and git-cvsserver.

*if you still read this* *wow*

Please discuss my arguments and come to the conclusions to
RESO/WONT-FIX testing git-cvsserver, make a clean cut and remove
this bug from the blockers of [TRACKER] portage migration to git.

My lengthy 2 cents.

[1] https://bugs.gentoo.org/333531
[2] https://bugs.gentoo.org/333699
[3] https://bugs.gentoo.org/333705#c2
- --
Gentoo Dev
http://xmw.de/
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iF4EAREIAAYFAk+82z0ACgkQknrdDGLu8JBUWAD/dmuqyES/mYDrMam+/txnHmgd
VaQaqwHMlwzzqQwbpY4A/0h+5Vp8sLbOE78k4SCaGE2dCQtmeOz0jd1YxkDzP+YW
=jXLQ
-END PGP SIGNATURE-



Another vote for clean cut.