Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Justin Lebar via lldb-commits]

jlebar added a comment.

In https://reviews.llvm.org/D22463#495211, @vladisld wrote:

> In https://reviews.llvm.org/D22463#494828, @jlebar wrote:
>
> > I think the general feeling is that most of us (myself included) would 
> > rather not learn a new tool if there's a simpler >alternative, such as a 
> > vanilla git workflow.
>
>
> Generally you're right, however learning how to use git-repo is much simpler 
> than managing the intricacies of git sub-modules (and Google's experience 
> with Android is a clear example of it). This is IMHO of cause.


Just to be clear, since it sounds like you haven't been following the llvm-dev 
discussion -- the alternative to git-repo is *not* submodules.  I agree that 
submodules are awful.  The alternative is a monolithic repository (monorepo) 
that contains, as a single git repository, the full history of all llvm 
subprojects.  Similar to https://github.com/llvm-project/llvm-project.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Daniel Berlin via lldb-commits]

On Mon, Jul 25, 2016 at 1:03 PM, Vlad Dovlekaev via llvm-commits <
llvm-comm...@lists.llvm.org> wrote:

> vladisld added a comment.
>
> In https://reviews.llvm.org/D22463#494828, @jlebar wrote:
>
> > I think the general feeling is that most of us (myself included) would
> rather not learn a new tool if there's a simpler >alternative, such as a
> vanilla git workflow.
>
>
> Generally you're right, however learning how to use git-repo is much
> simpler than managing the intricacies of git sub-modules (and Google's
> experience with Android is a clear example of it).


Just to make some points, as a guy who watched repo being designed in the
office next to me:
Sub modules were completely and totally unusable for Google at that point,
and something was needed sooner.
There is no good reason for android to change now, so they don't.

Note that repo does not solve the issue of cross-repo atomic commits in any
way, shape, or form, and in fact, this is the biggest complaint ;)

In fact, the gerrit team (same people who produce repo) uses submodules for
pseudo-atomic cross-repo commits.
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Chris Bieneman via lldb-commits]

beanz added a subscriber: beanz.
beanz added a comment.

@rengolin, thank you for putting this all together. It is very well thought 
out, and I really like the shape it took. I have a few minor nitpick comments 
inline.

Thanks,
-Chris



Comment at: docs/Proposals/GitHub.rst:58
@@ +57,3 @@
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN 
integration,

Not to be pedantic here, but do we actually /know/ that most LLVM developers 
use Git? I suspect that most do, but I don't think we actually have any 
metrics, so we should avoid using declarative language here. Maybe change 
'most' to 'many' here and below.


Comment at: docs/Proposals/GitHub.rst:87
@@ +86,3 @@
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and equality in our community.
+

In lists you should generally keep the same verb conjugations. Maybe repharse 
this list to something more like:


> ... hosting developer meetings, sponsoring disadvantaged people... and 
> fostering diversity and equality in our community.




https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Justin Lebar via lldb-commits]

jlebar added a comment.

In https://reviews.llvm.org/D22463#494568, @vladisld wrote:

> Have the alternatives to sub-modules and monolithic repository been discussed 
> ?


Hi, Vlad.

Please see the ongoing thread in llvm-dev, entitled "[RFC] One or many git 
repositories?".

Tools such as git-repo, have been discussed briefly.  I think the general 
feeling is that most of us (myself included) would rather not learn a new tool 
if there's a simpler alternative, such as a vanilla git workflow.  But if you 
would like to discuss more, that thread is the right place.

-Justin


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Justin Lebar via lldb-commits]

jlebar added a comment.

Hi, Renato.

Just to explain why I'm going to go forward with this RFC about a monolithic 
repository: From speaking with some top contributors on IRC, I have heard that 
they feel that the discussion of whether to move to git has been conflated with 
the discussion of how the git repository should be set up.  So there is a 
sizable set of important individuals who don't feel that this question has been 
considered sufficiently.

I would love not to alienate you by asking this question, but I understand if I 
already have.  If so, I sincerely apologize.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added a comment.

In https://reviews.llvm.org/D22463#489483, @jlebar wrote:

> Again, we can make this work with submodules, but it's a giant pain, see my 
> earlier comment.


(...)

> I've read as many of these as I can find in the past few hours, and every 
> argument I have found is, in my evaluation, very likely overblown or 
> incorrect.


We've heard both sides making equal claims. People work differently.

> The critical point is that it's trivial to use sparse checkouts to make the 
> monolithic repository behave identically to separate repos.  But it is 
> impossible, as far as I'm aware, to make separate repos behave like a 
> monolithic repository.  So the monolithic repository is strictly more 
> powerful.


LLVM is *not* a single project, but a large selection of smaller ones that 
*are* used independently by the *majority* of users. It may tax you more than 
others, but it will tax the majority less than today's solution.

This is not about finding the best possible way for everyone, since that's 
clearly impossible. This is about finding the least horrible solution for the 
majority.

> The e-mail you sent out two days ago said two weeks.  Can you give me a bit 
> more than three days?


Moving to Git has been in discussion for at least 2 years.

This time round, my first email with a concrete proposal to migrate was 2nd 
June. We had so far 320+ emails about the subject since, and the overwhelming 
majority is in favour to move to Git and a large part is *content* with 
sub-modules. Counter proposals were presented (including a monolithic 
repository) and were all shot down by the community (not me).

This is not the time to be second guessing ourselves. I'll be finishing this 
proposal this week and asking the foundation to put up a survey as soon as 
possible.

> But.  I would ask you to please give me a few days to work with the community 
> to dig in to this specific question.  If I am right, it will be a boon for 
> all of us every time we type a command that starts with "git".  And if I'm 
> wrong, I'll buy you a well-deserved beer or three, and we'll forget it and 
> move on.


A monolithic repository was proposed and discredited by the community. I can't 
vouch for it myself (in the interest of progress), but we *will* allow people 
to add comments on the survey. If there is a sound opposition to sub-modules in 
the survey, and a solid proposal to use a monolithic repo instead, we'll go to 
the next cycle, in which case, I'll politely step down and let other people in 
charge (whomever wants it).

All in all, we (for any definition of "we") are not going to force anyone to do 
anything they don't want. But as a community, we really should be thinking 
about the whole process, not just a single use case.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added a comment.

You will not be required to use submodules at all, as we'll all use the 
individual projects, like we have always been. I don't understand why people 
keep going back to it.

Having a single repository was part of the original proposal for years, and 
every time it was shot down as impractical.

Indeed, this is not the place for such discussion, but unless you can finish 
the discussion thus week, I suggest you make you point clear in the survey 
instead of delaying the process.

I'm not pushing for *my* solution. Thus *has* been discussed already to 
exhaustion. The current agreement was that we'd do a survey on the proposal, 
and that's what we need to do. Anything else will just send us back to square 
one and I seriously don't have the stamina to keep going round in circles.

Ie. Please, try to be considerate.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Justin Lebar via lldb-commits]

jlebar added a comment.

FYI after talking to Chandler, I'm going to write up a separate proposal for 
the one-repository thing and send it to the list tomorrow.  The suggestion was 
that this phabricator thread isn't the right place to have this discussion.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Justin Lebar via lldb-commits]

jlebar added a subscriber: jlebar.
jlebar added a comment.

I'm sure you all have thought about this more than I have, and I apologize if 
this has been brought up before because I haven't been following the thread 
closely.  But I am not convinced by this document that using subrepositories 
beats using a single git repo.

I see two reasons here for using subrepos as opposed to one big repository.

1. Subrepos mirror our current scheme.
2. Subrepos let people check out only the bits of llvm that they want.

I don't find either of these particularly compelling, compared to the 
advantages of one-big-repo (discussed below).  Taking them in turn:

1. Although subrepos would mirror our current scheme, it's going to be 
different *enough* that existing tools are going to have to change either way.  
In particular, the svn view of the master repository is not going to be useful 
for anything.  I tried `svn checkout 
https://github.com/chapuni/llvm-project-submodule`, and the result was 
essentially an empty repository.

2. It's true that subrepos let people check out only the bits that they want.  
But disk space and bandwidth are very cheap today, and LLVM is not as large as 
one might think.  My copy of https://github.com/llvm-project/llvm-project, 
which includes *everything* is 2.5G, whereas my copy of just llvm is 626M.

  Given that a release build of llvm and clang is ~3.5G, a 2.5G source checkout 
doesn't seem at all unreasonable to me.

  If it's really problematic, you can do a shallow checkout, which would take 
the contains-everything repo from 2.5G to 1.3G.  Moreover if it's *really* a 
problem, you can mirror the subdir of llvm that you care about.  Maybe the LLVM 
project could maintain said mirrors for some of the small subrepos that are 
often used independently.

So what's the advantage of using one big repository?  The simple answer is: 
Have you ever *tried* using git submodules?  :)

Submodules make everything more complicated.  Here's an example that I hope 
proves the point.  Suppose you want to commit your current work and switch to a 
new clean branch off head.  You make some changes there, then come back to your 
current work.  And let's assume that all of your changes are to clang only.

  # Commit current work, switch to a clean branch off head, then switch back.
  
  # One big repo: 
  $ git commit  # on old-branch
  $ git fetch
  $ git checkout -b new-branch origin/master
  # Hack hack hack...
  $ git commit
  $ git checkout old-branch
  
  # Submodules, attempt 1:
  $ cd clang
  $ git commit  # on old-branch
  $ git fetch
  $ git checkout -b new-branch origin/master
  # Also have to update llvm...
  $ cd ../llvm
  $ git fetch
  $ git checkout origin/master
  $ cd ../clang
  # Hack hack hack
  $ git commit
  
  # Now we're ready to switch back to old-branch, but...it's not going to work.
  # When we committed our old branch, we didn't save the state of our llvm
  # checkout.  So in particular we don't know which revision to roll it back to.
  
  # Let's try again.
  # Submodules, attempt 2:
  $ cd clang
  $ git commit  # on old-branch
  $ cd ..
  $ git checkout -b old-branch # in master repo
  $ git commit
  
  # Now we have two branches called "old-branch": One in the master repo, and 
one
  # in the clang submodules.  Now let's fetch head.
  
  $ git fetch  # in master repo
  $ git checkout -b new-branch origin/master
  $ git submodule update
  $ cd clang
  $ git checkout -b new-branch
  # Hack hack hack
  $ git commit  # in submodule
  $ cd ..
  $ git commit  # in master repo
  
  # Now we're ready to switch back.
  
  $ git checkout old-branch  # in master repo
  $ git submodule update

For those keeping track at home, this is 5 git commands with the big repo, and 
15 commands (11 git commands) in the submodules world.

Above we assumed that all of our changes were only to clang.  If we're making 
changes to both llvm and clang (say), the one-big-repo workflow remains 
identical, but the submodules workflow becomes even more complicated.

I'm sure people who are better at git than I can golf the above commands, but 
I'll suggest that I'm an above-average git user, so this is probably a 
lower-than-average estimate for the number of git commands (particularly `git 
help` :).  git is hard enough as-is; using submodules like this is asking a lot.

Similarly, I'm sure much of this can be scripted, but...seriously?  :)

Sorry for the wall of text.  tl;dr: One big repo doesn't actually cost that 
much, and that cost is dwarfed by the cost to humans of using submodules as 
proposed.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Robinson, Paul via lldb-commits]

> > I think we could emulate any pre-commit hook we like via GitHub
> > WebHooks by having two repositories: llvm and llvm-staging (say).
> >
> > People push to llvm-staging, which notifies some LLVM server we own.
> > That does basic sanity checks and pushes to llvm proper if passed.
> 
> I think that would be terrible in practice, for instance how do you handle
> situations like:
> 
> 1) Dev A push commit A
> 2) Dev B push commit B that changes some lines close to the one changed by
> commit A
> 3) sanity check fails on commit A, but llvm-staging contains A and B and
> can’t get rid of A easily because B would not apply without A.
> 
> At this point Dev B gets an email (or other mechanism, I don’t know what
> you imagined) telling that his changed are rejected for no good reason.
> 
> Also reverting to a state "before A” on llvm-staging would mean that that
> the history would be rewritten and everyone that pulled/fetched in the
> meantime would suffer .
> 
> If we want to go towards pre-check using staging, I believe we should work
> with pull-request (we’d still have the issue of conflicting PR, but I
> don’t believe it’d be that bad in practice).
> That’d be welcome, but that’d also be a whole other story to setup and
> maintain!
> 
> —
> Mehdi

Hear hear.  The schemes to handle this that I'm aware of do look more like
pull requests.  You submit your change to the pre-qualification queue.
If it rebases cleanly and passes pre-qual, your change becomes the new HEAD.
If it doesn't rebase cleanly or fails pre-qual, your change gets bounced.

A pull-request-like model also helps avoid the rebase-build-test-push(fail)
loop that you can get into with a very active git-based project.  This is
a mechanical task best suited to automation rather than wasting valuable
developer time on it.  But I suspect GitHub does not have anything like
this OOB so it would be an enhancement for a later time.
--paulr

P.S. At Sony we are building a system with a "staging" step but it's not
for our own work... more of a "flood control" dam.  :-)

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin updated this revision to Diff 64469.

https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,254 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from our own
+hosted Subversion to GitHub. Below are the financial and technical arguments as
+to why we need such a move and how will people (and validation infrastructure)
+continue to work with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+* Outline of the reasons to move to Git and GitHub
+* Description on what the work flow will look like (compared to SVN)
+* Remaining issues and potential problems
+* The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *free*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+
+* Commit, squash, merge, fork locally without any penalty to the server
+* Add as many branches as necessary to allow for multiple threads of development
+* Collaborate with peers directly, even without access to the Internet
+* Have multiple trees without multiplying disk space.
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide free code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and equality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people that still have/want to use SVN infrastructure and tooling can
+slowly migrate or even stay working as if it was an SVN repository (including
+read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but GitHub
+has two additional features that would be beneficial to the migration plan as
+well as the community already settled there.
+
+
+What will the new workflow look like
+
+
+In order to move version control, we need to make sure that 

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin updated this revision to Diff 64468.
rengolin added a comment.

Formatting issues (bullet points)


https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,253 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from our own
+hosted Subversion to GitHub. Below are the financial and technical arguments as
+to why we need such a move and how will people (and validation infrastructure)
+continue to work with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+* Outline of the reasons to move to Git and GitHub
+* Description on what the work flow will look like (compared to SVN)
+* Remaining issues and potential problems
+* The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *free*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+* Commit, squash, merge, fork locally without any penalty to the server
+* Add as many branches as necessary to allow for multiple threads of development
+* Collaborate with peers directly, even without access to the Internet
+* Have multiple trees without multiplying disk space.
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide free code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and equality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people that still have/want to use SVN infrastructure and tooling can
+slowly migrate or even stay working as if it was an SVN repository (including
+read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but GitHub
+has two additional features that would be beneficial to the migration plan as
+well as the community already settled there.
+
+
+What will the new workflow look like
+
+

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin updated this revision to Diff 64467.
rengolin added a comment.

More updates, following recent comments.


https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,253 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from our own
+hosted Subversion to GitHub. Below are the financial and technical arguments as
+to why we need such a move and how will people (and validation infrastructure)
+continue to work with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+ * Outline of the reasons to move to Git and GitHub
+ * Description on how the work flow will look like (compared to SVN)
+ * Remaining issues and potential problems
+ * The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *free*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+ * Commit, squash, merge, fork locally without any penalty to the server
+ * Add as many branches as necessary to allow for multiple threads of development
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space.
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide free code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and equality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people that still have/want to use SVN infrastructure and tooling can
+slowly migrate or even stay working as if it was an SVN repository (including
+read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but GitHub
+has two additional features that would be beneficial to the migration plan as
+well as the community already settled there.
+
+
+What will the new workflow look like

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:68
@@ +67,3 @@
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space, multiple concurrent 
builds
+

vsk wrote:
> What do you mean by multiple concurrent builds?
With git worktree you can work on source code and build different things at the 
same time, but I guess this is a specific use case which is only made "easier" 
in git. I'll remove this extra comment.


Comment at: docs/Proposals/GitHub.rst:78
@@ +77,3 @@
+
+GitHub, like GitLab and BitBucket, provide FREE code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure 
that

dberris wrote:
> nit: I see you use FREE in caps but this instance isn't *FREE* (as opposed to 
> the first mention above) -- consider making it consistent? Either remove the 
> emphasis (just "free") or emphasise consistently?
good point.


Comment at: docs/Proposals/GitHub.rst:86
@@ +85,3 @@
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and quality in our community.
+

dberris wrote:
> Did you mean "diversity and equality" instead of "diversity and quality" here?
indeed, fixed.


Comment at: docs/Proposals/GitHub.rst:102
@@ +101,3 @@
+
+How will the new workflow look like
+===

delcypher wrote:
> s/How will/What will/
ok


Comment at: docs/Proposals/GitHub.rst:110-112
@@ +109,5 @@
+
+As with the current SVN and Git repositories, each project will be separate,
+on its own, and people will also be able to check them out in the same way
+they're already doing today.
+

dberris wrote:
> Consider rewording this sentence -- it's a little too long and is trying to 
> say too many things.
> 
> Perhaps something like:
> 
> "Each LLVM project will continue to be hosted as separate GitHub repositories 
> under a single GitHub organisation. Users can continue to choose to use 
> either SVN or Git to access the repositories to suit their current workflow."
Much better, thanks!


Comment at: docs/Proposals/GitHub.rst:136
@@ +135,3 @@
+
+We will need additional server hooks to avoid non-fast-forwards commits (ex.
+merges, forced pushes, etc) in order to keep the linearity of the history.

delcypher wrote:
> @rengolin : I know GitHub enterprise has a "protected branch" feature to 
> prevent forced pushed ( 
> https://github.com/blog/2051-protected-branches-and-required-status-checks ). 
> You might want to speak to them to see if they can offer us that feature. I 
> think there's a limited support to do a merge as a squash and rebase too ( 
> https://github.com/blog/2141-squash-your-commits )
I'm asking about protected branches (it was proposed earlier, but I can't find 
any info on it).

But we don't want to squash people's commits. They can do that on their own 
before commit.


Comment at: docs/Proposals/GitHub.rst:185
@@ +184,3 @@
+
+Are there any other upstream systems that could be affected?
+

dberris wrote:
> vsk wrote:
> > Yes, the `llvmlab bisect` functionality is affected. IMO it is invaluable 
> > for bug triage. Could you add some kind of reassurance that initially, 
> > updating it for the new VC model will just require pointing it to github's 
> > SVN view?
> Probably worth mentioning how Phabricator will need to be updated to 
> integrate with the GitHub repository once the canonical repo is changed.
Adding llvmlab bisect and Phab to the list. Both should be trivial.


Comment at: docs/Proposals/GitHub.rst:209
@@ +208,3 @@
+   well as a webhook to update the umbrella project (see below).
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account, with all necessary hooks (history, update, merges).

jyknight wrote:
> mehdi_amini wrote:
> > rengolin wrote:
> > > mehdi_amini wrote:
> > > > > Pre-commit hooks
> > > > 
> > > > Won't handle the update of the umbrella.
> > > > 
> > > > > Post-commit hooks
> > > > 
> > > > Can't handle the update of the umbrella *because of GitHub*, this could 
> > > > be possible with our own hosting of git for instance.
> > > > 
> > > Pre-commit hooks are not designed to update the umbrella. Webhooks will 
> > > be able to update the umbrella with a small external service, as proposed 
> > > in the IRC.
> > That's why I asked it to be clearly mentioned somewhere else that on top of 
> > "hooks" hosted by github, we will need to maintain our own webservice on 
> > our own server to answer updates from theses hooks and update the umbrella, 
> > because that's a non-negligible drawback in face of the motivation exposed 
> > in the "Why move at all?" section. 
> > Right now the document 

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Filipe Cabecinhas via lldb-commits]

filcab added a comment.

Thanks a lot for working on this!

Filipe


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Mehdi Amini via lldb-commits]

> On Jul 18, 2016, at 8:23 PM, Tim Northover via cfe-commits 
>  wrote:
> 
>>> Can't handle the update of the umbrella *because of GitHub*, this could be 
>>> possible with our own hosting of git for instance.
>>> 
>> Pre-commit hooks are not designed to update the umbrella. Webhooks will be 
>> able to update the umbrella with a small external service, as proposed in 
>> the IRC.
> 
> I think we could emulate any pre-commit hook we like via GitHub
> WebHooks by having two repositories: llvm and llvm-staging (say).
> 
> People push to llvm-staging, which notifies some LLVM server we own.
> That does basic sanity checks and pushes to llvm proper if passed.

I think that would be terrible in practice, for instance how do you handle 
situations like:

1) Dev A push commit A
2) Dev B push commit B that changes some lines close to the one changed by 
commit A
3) sanity check fails on commit A, but llvm-staging contains A and B and can’t 
get rid of A easily because B would not apply without A.

At this point Dev B gets an email (or other mechanism, I don’t know what you 
imagined) telling that his changed are rejected for no good reason.

Also reverting to a state "before A” on llvm-staging would mean that that the 
history would be rewritten and everyone that pulled/fetched in the meantime 
would suffer .

If we want to go towards pre-check using staging, I believe we should work with 
pull-request (we’d still have the issue of conflicting PR, but I don’t believe 
it’d be that bad in practice).
That’d be welcome, but that’d also be a whole other story to setup and maintain!

— 
Mehdi


> 
> It has disadvantages (no instant "success" feedback being the obvious
> one), but would allow us to vet commits with relatively little
> overhead (as James says, running a server responding to webhooks is a
> very different proposition from one hammered by hundreds of developers
> daily).
> 
> I'm not strongly in favour of this, just thought I'd mention it as a
> possibility.
> 
> Tim.
> ___
> cfe-commits mailing list
> cfe-comm...@lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Dean Michael Berris via lldb-commits]

dberris added a subscriber: dberris.
dberris added a comment.

Mostly wording comments, thank you for writing this up!



Comment at: docs/Proposals/GitHub.rst:78
@@ +77,3 @@
+
+GitHub, like GitLab and BitBucket, provide FREE code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure 
that

nit: I see you use FREE in caps but this instance isn't *FREE* (as opposed to 
the first mention above) -- consider making it consistent? Either remove the 
emphasis (just "free") or emphasise consistently?


Comment at: docs/Proposals/GitHub.rst:86
@@ +85,3 @@
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and quality in our community.
+

Did you mean "diversity and equality" instead of "diversity and quality" here?


Comment at: docs/Proposals/GitHub.rst:110-112
@@ +109,5 @@
+
+As with the current SVN and Git repositories, each project will be separate,
+on its own, and people will also be able to check them out in the same way
+they're already doing today.
+

Consider rewording this sentence -- it's a little too long and is trying to say 
too many things.

Perhaps something like:

"Each LLVM project will continue to be hosted as separate GitHub repositories 
under a single GitHub organisation. Users can continue to choose to use either 
SVN or Git to access the repositories to suit their current workflow."


Comment at: docs/Proposals/GitHub.rst:185
@@ +184,3 @@
+
+Are there any other upstream systems that could be affected?
+

Probably worth mentioning how Phabricator will need to be updated to integrate 
with the GitHub repository once the canonical repo is changed.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Frederic Riss via lldb-commits]

friss added inline comments.


Comment at: docs/Proposals/GitHub.rst:220
@@ +219,3 @@
+8. Tell people living downstream to pick up commits from the official git
+   repository.
+9. Give things time to settle. We could play some games like disabling the SVN

vsk wrote:
> This is tricky. CC'ing @friss to comment on how we'd need to update our 
> internal auto-merger.
This is not an issue. We were already consuming the llvm.org git repos, so we'd 
just need to point to the new git repos (given the git history is preserved, I 
don't think I've read this anywhere in this document, but I think it's safe to 
make this assumption)


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Tim Northover via lldb-commits]

>> Can't handle the update of the umbrella *because of GitHub*, this could be 
>> possible with our own hosting of git for instance.
>>
> Pre-commit hooks are not designed to update the umbrella. Webhooks will be 
> able to update the umbrella with a small external service, as proposed in the 
> IRC.

I think we could emulate any pre-commit hook we like via GitHub
WebHooks by having two repositories: llvm and llvm-staging (say).

People push to llvm-staging, which notifies some LLVM server we own.
That does basic sanity checks and pushes to llvm proper if passed.

It has disadvantages (no instant "success" feedback being the obvious
one), but would allow us to vet commits with relatively little
overhead (as James says, running a server responding to webhooks is a
very different proposition from one hammered by hundreds of developers
daily).

I'm not strongly in favour of this, just thought I'd mention it as a
possibility.

Tim.
___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Vedant Kumar via lldb-commits]

vsk added subscribers: friss, vsk.
vsk added a comment.

@rengolin thanks for putting this together! I chimed in with some comments 
in-line.



Comment at: docs/Proposals/GitHub.rst:68
@@ +67,3 @@
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space, multiple concurrent 
builds
+

What do you mean by multiple concurrent builds?


Comment at: docs/Proposals/GitHub.rst:185
@@ +184,3 @@
+
+Are there any other upstream systems that could be affected?
+

Yes, the `llvmlab bisect` functionality is affected. IMO it is invaluable for 
bug triage. Could you add some kind of reassurance that initially, updating it 
for the new VC model will just require pointing it to github's SVN view?


Comment at: docs/Proposals/GitHub.rst:220
@@ +219,3 @@
+8. Tell people living downstream to pick up commits from the official git
+   repository.
+9. Give things time to settle. We could play some games like disabling the SVN

This is tricky. CC'ing @friss to comment on how we'd need to update our 
internal auto-merger.


Comment at: docs/Proposals/GitHub.rst:233
@@ +232,3 @@
+
+10. Collect peoples GitHub account information, adding them to the project.
+11. Switch SVN repository to read-only and allow pushes to the GitHub 
repository.

delcypher wrote:
> GitHub organizations support the notion of teams which can each have 
> different permissions (for example you'll want to make sure only the right 
> people have admin access and give the rest write/read access). You could also 
> make a team per project so that write access in one project does not give 
> write access to another. I think it would be good to decide on how teams will 
> be organized and state this in the document.
I think this is an important discussion to have once the move is under-way. I 
don't think finer-grained write privileges should be a part of this proposal 
since it's (1) a separate issue and (2) not *just* an artifact of 
llvm-project's svn structure (i.e there are good reasons to keep the current 
permissions model in place). 


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Jonathan Roelofs via lldb-commits]

jroelofs added a subscriber: jroelofs.


Comment at: docs/Proposals/GitHub.rst:127
@@ +126,3 @@
+* The projects' repositories will remain identical, with a new address 
(GitHub).
+* They'll continue to have SVN RW access, but will also gain Git RW access.
+* The linear history can still be accessed in the (RO) submodule meta project,

rengolin wrote:
> compnerd wrote:
> > jroelofs wrote:
> > > Do you mean `s/SVN RW access/SVN RO access/` here?
> > I believe @rengolin is referring to the final state here.  I agree that the 
> > current phrasing makes it hard to follow.
> No, I actually mean SVN RW access. GitHub's SVN view does allow write access 
> to the Git repos via "svn commit".
Ah, I didn't catch that part. Cool.


Comment at: docs/Proposals/GitHub.rst:136
@@ +135,3 @@
+Essentially, we're adding Git RW access in addition to the already existing
+structure, with all the additional benefits of it being in GitHub.
+

rengolin wrote:
> compnerd wrote:
> > jroelofs wrote:
> > > Need to clarify here whether *write* access through SVN will be going 
> > > away. If I understand the proposal correctly, it will go away, but this 
> > > section makes it sound like it's staying.
> > The way that I read the nutshell is that it would potentially continue to 
> > exist, just at a different address.
> Our SVN server will die, SVN access will continue via GitHub.
Ah, ok.


Repository:
  rL LLVM

https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Paul Robinson via lldb-commits]

probinson added a subscriber: probinson.


Comment at: docs/Proposals/GitHub.rst:141
@@ +140,3 @@
+has commit access to our current repository. In the future, you only need to
+provide the GitHub user to be granted access.
+

This reads a little bit like "we will create a GitHub account for you if you 
don't have one" but I suspect people actually need to create their own GitHub 
accounts first.  (We're not all on GitHub already!)


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Mehdi AMINI via lldb-commits]

mehdi_amini added a subscriber: mehdi_amini.


Comment at: docs/Proposals/GitHub.rst:122
@@ +121,3 @@
+of understanding the *sequence* in which commits were added by using the
+``git rev-list --count hash`` or ``git describe hash`` commands.
+

filcab wrote:
> How easy will it be to clone the "aggregated" repo, and then get some (but 
> not all) of the submodules?
I expect the umbrella repo to come with scripts for that.


Comment at: docs/Proposals/GitHub.rst:129
@@ +128,3 @@
+* The linear history can still be accessed in the (RO) submodule meta project.
+* Individual projects' history will be broken (linear, but local), and we need
+  the umbrella project (using submodules) to have the same view as we had in 
SVN.

This sentence is not clear: "Individual projects' history will be broken", I 
don't see what's "broken".


Comment at: docs/Proposals/GitHub.rst:168
@@ +167,3 @@
+additional benefits.
+
+2. Which tools will stop working?

I think that's covered line 136-137.


Comment at: docs/Proposals/GitHub.rst:173
@@ +172,3 @@
+use LNT with the SVN-View, but it would be best to move it to Git once and for
+all.
+

I don't think so: LNT is not dependent on SVN history. It is dependent on a 
single revision number cross-repository and cross-branches. This is something 
that the umbrella projects "could" provide.


Comment at: docs/Proposals/GitHub.rst:198
@@ +197,3 @@
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account.
+4. Make sure bisecting with llvm-project works.

Uh, this point is not clear: there will a need for us to maintain a 
"non-trivial" hook on our server to handle this. This is not fleshed out in 
this document.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Mehdi AMINI via lldb-commits]

mehdi_amini added inline comments.


Comment at: docs/Proposals/GitHub.rst:209
@@ +208,3 @@
+   well as a webhook to update the umbrella project (see below).
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account, with all necessary hooks (history, update, merges).

> Pre-commit hooks

Won't handle the update of the umbrella.

> Post-commit hooks

Can't handle the update of the umbrella *because of GitHub*, this could be 
possible with our own hosting of git for instance.



https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Mehdi AMINI via lldb-commits]

mehdi_amini added inline comments.


Comment at: docs/Proposals/GitHub.rst:199
@@ +198,3 @@
+
+Here's a proposed plan:
+

Annoyingly my comment does no longer show-up next to the point it was referring 
to, it was about your third point:

> Make sure we have an llvm-project (with submodules) setup in the official 
> account.

I don't think how this project will be updated is explained (or even mentioned) 
anywhere.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Wink Saville via lldb-commits]

winksaville added a subscriber: winksaville.


Comment at: docs/Proposals/GitHub.rst:132
@@ +131,3 @@
+
+There is no need to additional tags, flags and properties, nor of external
+services controlling the history, since both SVN and *git rev-list* can already

This could be worded a little better, may I suggest something like:

"There is no need for additional tags, flags, properties, or external ..."



https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:127
@@ +126,3 @@
+* The projects' repositories will remain identical, with a new address 
(GitHub).
+* They'll continue to have SVN RW access, but will also gain Git RW access.
+* The linear history can still be accessed in the (RO) submodule meta project,

jroelofs wrote:
> Do you mean `s/SVN RW access/SVN RO access/` here?
No, I actually mean SVN RW access. GitHub's SVN view does allow write access to 
the Git repos via "svn commit".


Comment at: docs/Proposals/GitHub.rst:136
@@ +135,3 @@
+Essentially, we're adding Git RW access in addition to the already existing
+structure, with all the additional benefits of it being in GitHub.
+

jroelofs wrote:
> Need to clarify here whether *write* access through SVN will be going away. 
> If I understand the proposal correctly, it will go away, but this section 
> makes it sound like it's staying.
Our SVN server will die, SVN access will continue via GitHub.


Repository:
  rL LLVM

https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Dan Liew via lldb-commits]

delcypher added a subscriber: delcypher.


Comment at: docs/Proposals/GitHub.rst:102
@@ +101,3 @@
+
+How will the new workflow look like
+===

s/How will/What will/


Comment at: docs/Proposals/GitHub.rst:136
@@ +135,3 @@
+
+We will need additional server hooks to avoid non-fast-forwards commits (ex.
+merges, forced pushes, etc) in order to keep the linearity of the history.

@rengolin : I know GitHub enterprise has a "protected branch" feature to 
prevent forced pushed ( 
https://github.com/blog/2051-protected-branches-and-required-status-checks ). 
You might want to speak to them to see if they can offer us that feature. I 
think there's a limited support to do a merge as a squash and rebase too ( 
https://github.com/blog/2141-squash-your-commits )


Comment at: docs/Proposals/GitHub.rst:233
@@ +232,3 @@
+
+10. Collect peoples GitHub account information, adding them to the project.
+11. Switch SVN repository to read-only and allow pushes to the GitHub 
repository.

GitHub organizations support the notion of teams which can each have different 
permissions (for example you'll want to make sure only the right people have 
admin access and give the rest write/read access). You could also make a team 
per project so that write access in one project does not give write access to 
another. I think it would be good to decide on how teams will be organized and 
state this in the document.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Mehdi AMINI via lldb-commits]

mehdi_amini added inline comments.


Comment at: docs/Proposals/GitHub.rst:209
@@ +208,3 @@
+   well as a webhook to update the umbrella project (see below).
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account, with all necessary hooks (history, update, merges).

rengolin wrote:
> mehdi_amini wrote:
> > > Pre-commit hooks
> > 
> > Won't handle the update of the umbrella.
> > 
> > > Post-commit hooks
> > 
> > Can't handle the update of the umbrella *because of GitHub*, this could be 
> > possible with our own hosting of git for instance.
> > 
> Pre-commit hooks are not designed to update the umbrella. Webhooks will be 
> able to update the umbrella with a small external service, as proposed in the 
> IRC.
That's why I asked it to be clearly mentioned somewhere else that on top of 
"hooks" hosted by github, we will need to maintain our own webservice on our 
own server to answer updates from theses hooks and update the umbrella, because 
that's a non-negligible drawback in face of the motivation exposed in the "Why 
move at all?" section. 
Right now the document does not acknowledge that AFAICT.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Mehdi AMINI via lldb-commits]

mehdi_amini added inline comments.


Comment at: docs/Proposals/GitHub.rst:208
@@ +207,3 @@
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account, with all necessary hooks (history, update, merges).
+4. Make sure bisecting with llvm-project works.

I'd like to see clearly mentioned somewhere else than in the plan that on top 
of "hooks" hosted by github, we will need to maintain our own webservice on our 
own server to answer updates from theses hooks and update the umbrella.
That's a non-negligible drawback in face of the motivation exposed in the "Why 
move at all?" section.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Jonathan Roelofs via lldb-commits]

jroelofs added inline comments.


Comment at: docs/Proposals/GitHub.rst:127
@@ +126,3 @@
+* The projects' repositories will remain identical, with a new address 
(GitHub).
+* They'll continue to have SVN RW access, but will also gain Git RW access.
+* The linear history can still be accessed in the (RO) submodule meta project,

Do you mean `s/SVN RW access/SVN RO access/` here?


Comment at: docs/Proposals/GitHub.rst:136
@@ +135,3 @@
+Essentially, we're adding Git RW access in addition to the already existing
+structure, with all the additional benefits of it being in GitHub.
+

Need to clarify here whether *write* access through SVN will be going away. If 
I understand the proposal correctly, it will go away, but this section makes it 
sound like it's staying.


Repository:
  rL LLVM

https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[James Y Knight via lldb-commits]

jyknight added a subscriber: jyknight.


Comment at: docs/Proposals/GitHub.rst:209
@@ +208,3 @@
+   well as a webhook to update the umbrella project (see below).
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account, with all necessary hooks (history, update, merges).

mehdi_amini wrote:
> rengolin wrote:
> > mehdi_amini wrote:
> > > > Pre-commit hooks
> > > 
> > > Won't handle the update of the umbrella.
> > > 
> > > > Post-commit hooks
> > > 
> > > Can't handle the update of the umbrella *because of GitHub*, this could 
> > > be possible with our own hosting of git for instance.
> > > 
> > Pre-commit hooks are not designed to update the umbrella. Webhooks will be 
> > able to update the umbrella with a small external service, as proposed in 
> > the IRC.
> That's why I asked it to be clearly mentioned somewhere else that on top of 
> "hooks" hosted by github, we will need to maintain our own webservice on our 
> own server to answer updates from theses hooks and update the umbrella, 
> because that's a non-negligible drawback in face of the motivation exposed in 
> the "Why move at all?" section. 
> Right now the document does not acknowledge that AFAICT.
The maintenance of that service will be negligible compared to running a 
subversion installation.

I expect that we could set up the webhook as an AppEngine app, using Github's 
"Git Data" API to generate the new commit, and then to the first approximation 
never have to touch it again.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:209
@@ +208,3 @@
+   well as a webhook to update the umbrella project (see below).
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account, with all necessary hooks (history, update, merges).

mehdi_amini wrote:
> > Pre-commit hooks
> 
> Won't handle the update of the umbrella.
> 
> > Post-commit hooks
> 
> Can't handle the update of the umbrella *because of GitHub*, this could be 
> possible with our own hosting of git for instance.
> 
Pre-commit hooks are not designed to update the umbrella. Webhooks will be able 
to update the umbrella with a small external service, as proposed in the IRC.


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin updated this revision to Diff 64383.
rengolin added a comment.

Expand step 2 to make sure we don't forget about the safety hooks on each 
project as well as the webhook to update the umbrella project. This could turn 
out to be a buildbot, but makes no difference at this stage.


https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,242 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from our own
+hosted Subversion to GitHub. Below are the financial and technical arguments as
+to why we need such a move and how will people (and validation infrastructure)
+continue to work with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+ * Outline of the reasons to move to Git and GitHub
+ * Description on how the work flow will look like (compared to SVN)
+ * Remaining issues and potential problems
+ * The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *FREE*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+ * Commit, squash, merge, fork locally without any penalty to the server
+ * Add as many branches as necessary to allow for multiple threads of development
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space, multiple concurrent builds
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide FREE code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and quality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people that still have/want to use SVN infrastructure and tooling can
+slowly migrate or even stay working as if it was an SVN repository (including
+read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but 

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:200
@@ +199,3 @@
+
+Here's a proposed plan:
+

You can click on the "<<" button and it will show where it was first inserted. 
That's how I found out. :)

The hooks, AFAICS, will be added to the project initially, and won't need to be 
updated. Takumi's current repository is automatically updated, and IIRC, it's 
just a hook.

Assuming it's atomic and quick enough, would this process make much of a 
difference to the users?


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin updated this revision to Diff 64373.
rengolin added a comment.

Removing "broken" to describe the history, just explaining it'll be local.

Expanding to mention that hooks will need to be implemented in step 3.


https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,240 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from our own
+hosted Subversion to GitHub. Below are the financial and technical arguments as
+to why we need such a move and how will people (and validation infrastructure)
+continue to work with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+ * Outline of the reasons to move to Git and GitHub
+ * Description on how the work flow will look like (compared to SVN)
+ * Remaining issues and potential problems
+ * The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *FREE*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+ * Commit, squash, merge, fork locally without any penalty to the server
+ * Add as many branches as necessary to allow for multiple threads of development
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space, multiple concurrent builds
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide FREE code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and quality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people that still have/want to use SVN infrastructure and tooling can
+slowly migrate or even stay working as if it was an SVN repository (including
+read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but GitHub
+has two additional features that would be beneficial to the 

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:198
@@ +197,3 @@
+3. Make sure we have an llvm-project (with submodules) setup in the official
+   account.
+4. Make sure bisecting with llvm-project works.

mehdi_amini wrote:
> Uh, this point is not clear: there will a need for us to maintain a 
> "non-trivial" hook on our server to handle this. This is not fleshed out in 
> this document.
Good point. I added just a description to this topic, since this is covered 
elsewhere.


Comment at: docs/Proposals/GitHub.rst:174
@@ +173,3 @@
+us what's wrong and how to help them fix it.
+
+We also recommend people and companies to migrate to Git, for its many other

So, LNT migration plan could be:

1. Use GitHub's SVN view on llvm-proj-submods
2. Move to understand submods
3. Migrate all instances

Looks fairly orthogonal to me...


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin updated this revision to Diff 64371.
rengolin added a comment.

Second round of suggestions applied.


https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,239 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from our own
+hosted Subversion to GitHub. Below are the financial and technical arguments as
+to why we need such a move and how will people (and validation infrastructure)
+continue to work with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+ * Outline of the reasons to move to Git and GitHub
+ * Description on how the work flow will look like (compared to SVN)
+ * Remaining issues and potential problems
+ * The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *FREE*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+ * Commit, squash, merge, fork locally without any penalty to the server
+ * Add as many branches as necessary to allow for multiple threads of development
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space, multiple concurrent builds
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide FREE code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and quality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people that still have/want to use SVN infrastructure and tooling can
+slowly migrate or even stay working as if it was an SVN repository (including
+read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but GitHub
+has two additional features that would be beneficial to the migration plan as
+well as the community already settled there.
+
+
+How will the new workflow look like

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:8-9
@@ +7,4 @@
+
+This is a proposal to move our current revision control system from Subversion
+to GitHub. Below are the financial and technical arguments as to why we need
+such a move and how will people (and validation infrastructure) continue to 
work

emaste wrote:
> It seems pedantic, but I think we should try hard to avoid conflating Git and 
> GitHub. What about: "move our revision control system from //self-hosted// 
> Subversion to Git hosted by GitHub."
This is a summary of the whole proposal, which specifically dictates GitHub.

We're not proposing to move to some Git hosting anymore, but exactly GitHub, 
due to the constraints that we outline below. If we do not move to GitHub 
specifically, a lot of the assumptions below will be wrong, and this proposal 
can't be accepted.

There is a paragraph on why git, and another on why GitHub, and both are key 
points of this proposal.

I'll change "from Subversion" to "from our own hosted Subversion" to make that 
even more clear.


Comment at: docs/Proposals/GitHub.rst:93
@@ +92,3 @@
+Furthermore, GitHub has an *SVN view* 
(https://github.com/blog/626-announcing-svn-support)
+where people stuck with SVN infrastructure and tooling can slowly migrate or
+even stay working as if it was an SVN repository (including read-write access).

emaste wrote:
> Replace "stuck" with something neutral. "stuck" implies that everyone will 
> want to move but some may not be able to for technical or other reasons, but 
> some people actually prefer SVN.
good point.


Comment at: docs/Proposals/GitHub.rst:122
@@ +121,3 @@
+of understanding the *sequence* in which commits were added by using the
+``git rev-list --count hash`` or ``git describe hash`` commands.
+

filcab wrote:
> How easy will it be to clone the "aggregated" repo, and then get some (but 
> not all) of the submodules?
Checking out this project:

https://github.com/chapuni/llvm-project-submodule

Will return the references, not the sub modules. You have to "init" each 
sub-module independently, which achieves the same task as only checking out the 
SVN repos you need, with the correct numbering.


Comment at: docs/Proposals/GitHub.rst:130
@@ +129,3 @@
+* Individual projects' history will be broken (linear, but local), and we need
+  the umbrella project (using submodules) to have the same view as we had in 
SVN.
+

filcab wrote:
> I wouldn't call it broken.
> Won't it have the same end result as having a checkout per project and simply 
> updating them close to each other?
> 
> Basically, it won't be "any more broken" than using this method for updating:
> 
> ```
> #!/bin/bash
> for dir in llvm/{,tools/{clang,lld},projects/{libcxx,libcxxabi,compiler-rt}}; 
> do
>   # (cd $dir && svn up) # for SVN
>   (cd $dir && git checkout master && git pull) # for git
> done
> ```
No, it won't.

Checking out LLVM only skips all commits from all other repos. So, for example:

LNT 123
Clang 124
RT 125
LLVM 126

Then, "svn checkout 126" will be:

In LNT, 123 as 126
In Clang, 124 as 126
In RT, 125 as 126
In LLVM, 126 as 126

With the new SVN interface, each one of those commits will be referred to, 
locally, as 123, and 126 will not exist, because the "git rev-list --count" 
won't get as high as 126.

However, on the umbrella submodule project, because the sequence of the commits 
is guaranteed, the rev-list count will bring the correct numbering, the same as 
via the SVN interface, and thus be "just like SVN was".


Comment at: docs/Proposals/GitHub.rst:132
@@ +131,3 @@
+
+There is no need to additional tags, flags and properties, nor of external
+services controlling the history, since both SVN and *git rev-list* can already

winksaville wrote:
> This could be worded a little better, may I suggest something like:
> 
> "There is no need for additional tags, flags, properties, or external ..."
> 
Yup, changing on next round. Thanks!


Comment at: docs/Proposals/GitHub.rst:141
@@ +140,3 @@
+has commit access to our current repository. In the future, you only need to
+provide the GitHub user to be granted access.
+

probinson wrote:
> This reads a little bit like "we will create a GitHub account for you if you 
> don't have one" but I suspect people actually need to create their own GitHub 
> accounts first.  (We're not all on GitHub already!)
well, "you need to provide the GitHub user" should take care of that, but I'll 
try to re-write this to make it more clear.

> (We're not all on GitHub already!)

Are we not? Egregious! :D


Comment at: docs/Proposals/GitHub.rst:180
@@ +179,3 @@
+
+As soon as we decide to move, we'll have to set a date for the process to 
begin.
+

emaste wrote:
> This presents it 

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Filipe Cabecinhas via lldb-commits]

filcab added a subscriber: filcab.
filcab added a comment.

What about branches? I'm guessing we should expect the usual release branches. 
But will any person be able to create a branch? Will there be a policy, if this 
is the case? Is the policy enforceable?



Comment at: docs/Proposals/GitHub.rst:122
@@ +121,3 @@
+of understanding the *sequence* in which commits were added by using the
+``git rev-list --count hash`` or ``git describe hash`` commands.
+

How easy will it be to clone the "aggregated" repo, and then get some (but not 
all) of the submodules?


Comment at: docs/Proposals/GitHub.rst:130
@@ +129,3 @@
+* Individual projects' history will be broken (linear, but local), and we need
+  the umbrella project (using submodules) to have the same view as we had in 
SVN.
+

I wouldn't call it broken.
Won't it have the same end result as having a checkout per project and simply 
updating them close to each other?

Basically, it won't be "any more broken" than using this method for updating:

```
#!/bin/bash
for dir in llvm/{,tools/{clang,lld},projects/{libcxx,libcxxabi,compiler-rt}}; do
  # (cd $dir && svn up) # for SVN
  (cd $dir && git checkout master && git pull) # for git
done
```


https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin removed rL LLVM as the repository for this revision.
rengolin updated this revision to Diff 64334.
rengolin added a comment.

First round of changes reflecting reviews.


https://reviews.llvm.org/D22463

Files:
  docs/Proposals/GitHub.rst

Index: docs/Proposals/GitHub.rst
===
--- /dev/null
+++ docs/Proposals/GitHub.rst
@@ -0,0 +1,230 @@
+==
+Moving LLVM Projects to GitHub
+==
+
+Introduction
+
+
+This is a proposal to move our current revision control system from Subversion
+to GitHub. Below are the financial and technical arguments as to why we need
+such a move and how will people (and validation infrastructure) continue to work
+with a Git-based LLVM.
+
+There will be a survey pointing at this document when we'll know the community's
+reaction and, if we collectively decide to move, the time-frames. Be sure to make
+your views count.
+
+Essentially, the proposal is divided in the following parts:
+
+ * Outline of the reasons to move to Git and GitHub
+ * Description on how the work flow will look like (compared to SVN)
+ * Remaining issues and potential problems
+ * The proposed migration plan
+
+Why Git, and Why GitHub?
+
+
+Why move at all?
+
+
+The strongest reason for the move, and why this discussion started in the first
+place, is that we currently host our own Subversion server and Git mirror in a
+voluntary basis. The LLVM Foundation sponsors the server and provides limited
+support, but there is only so much it can do.
+
+The volunteers are not Sysadmins themselves, but compiler engineers that happen
+to know a thing or two about hosting servers. We also don't have 24/7 support,
+and we sometimes wake up to see that continuous integration is broken because
+the SVN server is either down or unresponsive.
+
+With time and money, the foundation and volunteers could improve our services,
+implement more functionality and provide around the clock support, so that we
+can have a first class infrastructure with which to work. But the cost is not
+small, both in money and time invested.
+
+On the other hand, there are multiple services out there (GitHub, GitLab,
+BitBucket among others) that offer that same service (24/7 stability, disk space,
+Git server, code browsing, forking facilities, etc) for the very affordable price
+of *FREE*.
+
+Why Git?
+
+
+Most new coders nowadays start with Git. A lot of them have never used SVN, CVS
+or anything else. Websites like GitHub have changed the landscape of open source
+contributions, reducing the cost of first contribution and fostering
+collaboration.
+
+Git is also the version control most LLVM developers use. Despite the sources
+being stored in an SVN server, most people develop using the Git-SVN integration,
+and that shows that Git is not only more powerful than SVN, but people have
+resorted to using a bridge because its features are now indispensable to their
+internal and external workflows.
+
+In essence, Git allows you to:
+ * Commit, squash, merge, fork locally without any penalty to the server
+ * Add as many branches as necessary to allow for multiple threads of development
+ * Collaborate with peers directly, even without access to the Internet
+ * Have multiple trees without multiplying disk space, multiple concurrent builds
+
+In addition, because Git seems to be replacing every project's version control
+system, there are many more tools that can use Git's enhanced feature set, so
+new tooling is much more likely to support Git first (if not only), than any
+other version control system.
+
+Why GitHub?
+---
+
+GitHub, like GitLab and BitBucket, provide FREE code hosting for open source
+projects. Essentially, they will completely replace *all* the infrastructure that
+we have today that serves code repository, mirroring, user control, etc.
+
+They also have a dedicated team to monitor, migrate, improve and distribute the
+contents of the repositories depending on region and load. A level of quality
+that we'd never have without spending money that would be better spent elsewhere,
+for example development meetings, sponsoring disadvantaged people to work on
+compilers and foster diversity and quality in our community.
+
+GitHub has the added benefit that we already have a presence there. Many
+developers use it already, and the mirror from our current repository is already
+set up.
+
+Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support)
+where people stuck with SVN infrastructure and tooling can slowly migrate or
+even stay working as if it was an SVN repository (including read-write access).
+
+So, any of the three solutions solve the cost and maintenance problem, but GitHub
+has two additional features that would be beneficial to the migration plan as
+well as the community already settled there.
+
+
+How 

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Saleem Abdulrasool via lldb-commits]

compnerd added inline comments.


Comment at: docs/Proposals/GitHub.rst:167
@@ +166,3 @@
+with the limited number of developers whose job will be to mainly merge
+thousands of patches a day.
+

rengolin wrote:
> compnerd wrote:
> > I don't fully understand how this is any different from today.  We have a 
> > core set of developers with commit access.  Others are encouraged to 
> > provide patches via email (or may use phabricator depending on the 
> > reviewer).  Once reviewed and accepted, one of the core developers still 
> > commits the change.  I just see this as a process change.
> > 
> > The person forks the repository on github, and creates a branch, and then a 
> > PR.  The PR is reviewed and once accepted, merged by one of the core 
> > developers.  It even implicitly handles authorship tracking which has 
> > currently been done in an adhoc fashion via the commit message.
> Today we all commit to SVN, which is linear. In GitHub, we'll be committing 
> to git. If we can have hooks forbidding merges, it'll remain linear, but then 
> pull requests will be blocked. Additional hooks will need to be in place 
> (please suggest all of them here and I'll update the doc).
I think that we should aim to preserve the linearity of history.  This would 
mean that we block non-fastforward commits (i.e. no merges, no force pushes).


Repository:
  rL LLVM

https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits

Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!

[Renato Golin via lldb-commits]

rengolin added inline comments.


Comment at: docs/Proposals/GitHub.rst:129
@@ +128,3 @@
+* The linear history can still be accessed in the (RO) submodule meta project,
+  Which will continue to have SVN access.
+

compnerd wrote:
> "Which will continue to have SVN access" is redundant given the previous 
> statement.
good point. I'll try and re-write those points to be more clear.


Comment at: docs/Proposals/GitHub.rst:155
@@ +154,3 @@
+But some people/companies might not be allowed to use GitHub or might have
+firewalls blocking certain websites.
+

compnerd wrote:
> GitHub does have HTTPS based connections.  It seems highly unlikely that this 
> is a real concern.  Companies would have to go out of their way to block 
> access specifically to github over SSH and HTTPS.
I have had this problem in China... Though no one has raised this issue, so 
I'll just remove and let people complain about this in the survey.


Comment at: docs/Proposals/GitHub.rst:167
@@ +166,3 @@
+with the limited number of developers whose job will be to mainly merge
+thousands of patches a day.
+

compnerd wrote:
> I don't fully understand how this is any different from today.  We have a 
> core set of developers with commit access.  Others are encouraged to provide 
> patches via email (or may use phabricator depending on the reviewer).  Once 
> reviewed and accepted, one of the core developers still commits the change.  
> I just see this as a process change.
> 
> The person forks the repository on github, and creates a branch, and then a 
> PR.  The PR is reviewed and once accepted, merged by one of the core 
> developers.  It even implicitly handles authorship tracking which has 
> currently been done in an adhoc fashion via the commit message.
Today we all commit to SVN, which is linear. In GitHub, we'll be committing to 
git. If we can have hooks forbidding merges, it'll remain linear, but then pull 
requests will be blocked. Additional hooks will need to be in place (please 
suggest all of them here and I'll update the doc).


Comment at: docs/Proposals/GitHub.rst:222
@@ +221,3 @@
+10. Collect peoples GitHub account information, give them push access. Ideally
+while still locking the GitHub repository somehow...
+11. Switch SVN repository to read-only and allow pushes to the GitHub 
repository.

compnerd wrote:
> Giving permissions to only the LLVM "project" is sufficient.  People can be 
> added to the LLVM "project" as collaborators and get access that way.  This 
> is similar to how Apple is managing swift for comparison.
That's what I meant but I will change the wording.


Repository:
  rL LLVM

https://reviews.llvm.org/D22463



___
lldb-commits mailing list
lldb-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits