Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jlebar added a comment. In https://reviews.llvm.org/D22463#495211, @vladisld wrote: > In https://reviews.llvm.org/D22463#494828, @jlebar wrote: > > > I think the general feeling is that most of us (myself included) would > > rather not learn a new tool if there's a simpler >alternative, such as a > > vanilla git workflow. > > > Generally you're right, however learning how to use git-repo is much simpler > than managing the intricacies of git sub-modules (and Google's experience > with Android is a clear example of it). This is IMHO of cause. Just to be clear, since it sounds like you haven't been following the llvm-dev discussion -- the alternative to git-repo is *not* submodules. I agree that submodules are awful. The alternative is a monolithic repository (monorepo) that contains, as a single git repository, the full history of all llvm subprojects. Similar to https://github.com/llvm-project/llvm-project. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
On Mon, Jul 25, 2016 at 1:03 PM, Vlad Dovlekaev via llvm-commits < llvm-comm...@lists.llvm.org> wrote: > vladisld added a comment. > > In https://reviews.llvm.org/D22463#494828, @jlebar wrote: > > > I think the general feeling is that most of us (myself included) would > rather not learn a new tool if there's a simpler >alternative, such as a > vanilla git workflow. > > > Generally you're right, however learning how to use git-repo is much > simpler than managing the intricacies of git sub-modules (and Google's > experience with Android is a clear example of it). Just to make some points, as a guy who watched repo being designed in the office next to me: Sub modules were completely and totally unusable for Google at that point, and something was needed sooner. There is no good reason for android to change now, so they don't. Note that repo does not solve the issue of cross-repo atomic commits in any way, shape, or form, and in fact, this is the biggest complaint ;) In fact, the gerrit team (same people who produce repo) uses submodules for pseudo-atomic cross-repo commits. ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
beanz added a subscriber: beanz. beanz added a comment. @rengolin, thank you for putting this all together. It is very well thought out, and I really like the shape it took. I have a few minor nitpick comments inline. Thanks, -Chris Comment at: docs/Proposals/GitHub.rst:58 @@ +57,3 @@ + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, Not to be pedantic here, but do we actually /know/ that most LLVM developers use Git? I suspect that most do, but I don't think we actually have any metrics, so we should avoid using declarative language here. Maybe change 'most' to 'many' here and below. Comment at: docs/Proposals/GitHub.rst:87 @@ +86,3 @@ +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and equality in our community. + In lists you should generally keep the same verb conjugations. Maybe repharse this list to something more like: > ... hosting developer meetings, sponsoring disadvantaged people... and > fostering diversity and equality in our community. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jlebar added a comment. In https://reviews.llvm.org/D22463#494568, @vladisld wrote: > Have the alternatives to sub-modules and monolithic repository been discussed > ? Hi, Vlad. Please see the ongoing thread in llvm-dev, entitled "[RFC] One or many git repositories?". Tools such as git-repo, have been discussed briefly. I think the general feeling is that most of us (myself included) would rather not learn a new tool if there's a simpler alternative, such as a vanilla git workflow. But if you would like to discuss more, that thread is the right place. -Justin https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jlebar added a comment. Hi, Renato. Just to explain why I'm going to go forward with this RFC about a monolithic repository: From speaking with some top contributors on IRC, I have heard that they feel that the discussion of whether to move to git has been conflated with the discussion of how the git repository should be set up. So there is a sizable set of important individuals who don't feel that this question has been considered sufficiently. I would love not to alienate you by asking this question, but I understand if I already have. If so, I sincerely apologize. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin accepted this revision. rengolin added a reviewer: rengolin. rengolin added a comment. This revision is now accepted and ready to land. I'm auto accepting this proposal, as it seems to have ran its course. The commit is r276097. If anyone has any additional comment/suggestion, please submit a new review. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added a comment. In https://reviews.llvm.org/D22463#489483, @jlebar wrote: > Again, we can make this work with submodules, but it's a giant pain, see my > earlier comment. (...) > I've read as many of these as I can find in the past few hours, and every > argument I have found is, in my evaluation, very likely overblown or > incorrect. We've heard both sides making equal claims. People work differently. > The critical point is that it's trivial to use sparse checkouts to make the > monolithic repository behave identically to separate repos. But it is > impossible, as far as I'm aware, to make separate repos behave like a > monolithic repository. So the monolithic repository is strictly more > powerful. LLVM is *not* a single project, but a large selection of smaller ones that *are* used independently by the *majority* of users. It may tax you more than others, but it will tax the majority less than today's solution. This is not about finding the best possible way for everyone, since that's clearly impossible. This is about finding the least horrible solution for the majority. > The e-mail you sent out two days ago said two weeks. Can you give me a bit > more than three days? Moving to Git has been in discussion for at least 2 years. This time round, my first email with a concrete proposal to migrate was 2nd June. We had so far 320+ emails about the subject since, and the overwhelming majority is in favour to move to Git and a large part is *content* with sub-modules. Counter proposals were presented (including a monolithic repository) and were all shot down by the community (not me). This is not the time to be second guessing ourselves. I'll be finishing this proposal this week and asking the foundation to put up a survey as soon as possible. > But. I would ask you to please give me a few days to work with the community > to dig in to this specific question. If I am right, it will be a boon for > all of us every time we type a command that starts with "git". And if I'm > wrong, I'll buy you a well-deserved beer or three, and we'll forget it and > move on. A monolithic repository was proposed and discredited by the community. I can't vouch for it myself (in the interest of progress), but we *will* allow people to add comments on the survey. If there is a sound opposition to sub-modules in the survey, and a solid proposal to use a monolithic repo instead, we'll go to the next cycle, in which case, I'll politely step down and let other people in charge (whomever wants it). All in all, we (for any definition of "we") are not going to force anyone to do anything they don't want. But as a community, we really should be thinking about the whole process, not just a single use case. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jlebar added a comment. In https://reviews.llvm.org/D22463#489461, @rengolin wrote: > You will not be required to use submodules at all, as we'll all use the > individual projects, like we have always been. I don't understand why people > keep going back to it. There is a key use case that is not supported by the current setup. This is something that I -- and basically anyone who works on an llvm project other than llvm itself -- do every day. It will be supported with submodules, but badly. It is not supported at all today, and it will not be supported at all in the proposed world if I don't use submodules. Maybe the people who keep coming back to this have not explained this use-case clearly, so let me try. The use-case is: Maintaining a branch which contains changes to clang (or your favorite subproject) and also is locked to a specific revision of llvm. That is, I can check out branch A or branch B of my changes to clang, and it automatically checks out a corresponding revision of llvm that is tied to the branch. Again, we can make this work with submodules, but it's a giant pain, see my earlier comment. It works with zero effort if we have a monolithic repository. This would be 'uge for anyone who works on clang (or any other subproject that's not llvm itself) and uses branches. > Having a single repository was part of the original proposal for years, and > every time it was shot down as impractical. I've read as many of these as I can find in the past few hours, and every argument I have found is, in my evaluation, very likely overblown or incorrect. There is strong evidence for this, in the single git repository that already exists (and includes the test-suite, so is much larger than what I propose), and also in the fact that adding everything to a single git repository will not more than ~double the size of the llvm git repo. (I'll have better numbers tomorrow, don't quote me on that just yet.) Moreover, the current setup of unrelated git repos can be *exactly duplicated* by making sparse checkouts of the monolithic repository. You can clone the big repo and then check out only the directories you want (so it's like the others never existed, beyond their presence in your .git packfiles). Or if you want to be able to check out different revisions of (say) clang and llvm, you can do that too: Clone the big repo and make two shallow working copies, one for llvm and the other for clang. You can even place the clang working copy in tools/clang of the llvm working copy, so it functions identically to the current setup in almost every way. The critical point is that it's trivial to use sparse checkouts to make the monolithic repository behave identically to separate repos. But it is impossible, as far as I'm aware, to make separate repos behave like a monolithic repository. So the monolithic repository is strictly more powerful. > Indeed, this is not the place for such discussion, but unless you can finish > the discussion thus week, I suggest you make you point clear in the survey > instead of delaying the process. The e-mail you sent out two days ago said two weeks. Can you give me a bit more than three days? > I'm not pushing for *my* solution. Thus *has* been discussed already to > exhaustion. The current agreement was that we'd do a survey on the proposal, > and that's what we need to do. Anything else will just send us back to square > one and I seriously don't have the stamina to keep going round in circles. > > Ie. Please, try to be considerate. I am very grateful for the work that you're doing here. I have participated in efforts very similar to this one in the past, and I appreciate how difficult and taxing they can be, and also how frustrating it can be to be see perfect be made the enemy of the good. In fact I quit my last job in part over friction created by a botched move to git. But. I would ask you to please give me a few days to work with the community to dig in to this specific question. If I am right, it will be a boon for all of us every time we type a command that starts with "git". And if I'm wrong, I'll buy you a well-deserved beer or three, and we'll forget it and move on. Does that sound agreeable to you? https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added a comment. You will not be required to use submodules at all, as we'll all use the individual projects, like we have always been. I don't understand why people keep going back to it. Having a single repository was part of the original proposal for years, and every time it was shot down as impractical. Indeed, this is not the place for such discussion, but unless you can finish the discussion thus week, I suggest you make you point clear in the survey instead of delaying the process. I'm not pushing for *my* solution. Thus *has* been discussed already to exhaustion. The current agreement was that we'd do a survey on the proposal, and that's what we need to do. Anything else will just send us back to square one and I seriously don't have the stamina to keep going round in circles. Ie. Please, try to be considerate. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jlebar added a comment. FYI after talking to Chandler, I'm going to write up a separate proposal for the one-repository thing and send it to the list tomorrow. The suggestion was that this phabricator thread isn't the right place to have this discussion. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jlebar added a subscriber: jlebar. jlebar added a comment. I'm sure you all have thought about this more than I have, and I apologize if this has been brought up before because I haven't been following the thread closely. But I am not convinced by this document that using subrepositories beats using a single git repo. I see two reasons here for using subrepos as opposed to one big repository. 1. Subrepos mirror our current scheme. 2. Subrepos let people check out only the bits of llvm that they want. I don't find either of these particularly compelling, compared to the advantages of one-big-repo (discussed below). Taking them in turn: 1. Although subrepos would mirror our current scheme, it's going to be different *enough* that existing tools are going to have to change either way. In particular, the svn view of the master repository is not going to be useful for anything. I tried `svn checkout https://github.com/chapuni/llvm-project-submodule`, and the result was essentially an empty repository. 2. It's true that subrepos let people check out only the bits that they want. But disk space and bandwidth are very cheap today, and LLVM is not as large as one might think. My copy of https://github.com/llvm-project/llvm-project, which includes *everything* is 2.5G, whereas my copy of just llvm is 626M. Given that a release build of llvm and clang is ~3.5G, a 2.5G source checkout doesn't seem at all unreasonable to me. If it's really problematic, you can do a shallow checkout, which would take the contains-everything repo from 2.5G to 1.3G. Moreover if it's *really* a problem, you can mirror the subdir of llvm that you care about. Maybe the LLVM project could maintain said mirrors for some of the small subrepos that are often used independently. So what's the advantage of using one big repository? The simple answer is: Have you ever *tried* using git submodules? :) Submodules make everything more complicated. Here's an example that I hope proves the point. Suppose you want to commit your current work and switch to a new clean branch off head. You make some changes there, then come back to your current work. And let's assume that all of your changes are to clang only. # Commit current work, switch to a clean branch off head, then switch back. # One big repo: $ git commit # on old-branch $ git fetch $ git checkout -b new-branch origin/master # Hack hack hack... $ git commit $ git checkout old-branch # Submodules, attempt 1: $ cd clang $ git commit # on old-branch $ git fetch $ git checkout -b new-branch origin/master # Also have to update llvm... $ cd ../llvm $ git fetch $ git checkout origin/master $ cd ../clang # Hack hack hack $ git commit # Now we're ready to switch back to old-branch, but...it's not going to work. # When we committed our old branch, we didn't save the state of our llvm # checkout. So in particular we don't know which revision to roll it back to. # Let's try again. # Submodules, attempt 2: $ cd clang $ git commit # on old-branch $ cd .. $ git checkout -b old-branch # in master repo $ git commit # Now we have two branches called "old-branch": One in the master repo, and one # in the clang submodules. Now let's fetch head. $ git fetch # in master repo $ git checkout -b new-branch origin/master $ git submodule update $ cd clang $ git checkout -b new-branch # Hack hack hack $ git commit # in submodule $ cd .. $ git commit # in master repo # Now we're ready to switch back. $ git checkout old-branch # in master repo $ git submodule update For those keeping track at home, this is 5 git commands with the big repo, and 15 commands (11 git commands) in the submodules world. Above we assumed that all of our changes were only to clang. If we're making changes to both llvm and clang (say), the one-big-repo workflow remains identical, but the submodules workflow becomes even more complicated. I'm sure people who are better at git than I can golf the above commands, but I'll suggest that I'm an above-average git user, so this is probably a lower-than-average estimate for the number of git commands (particularly `git help` :). git is hard enough as-is; using submodules like this is asking a lot. Similarly, I'm sure much of this can be scripted, but...seriously? :) Sorry for the wall of text. tl;dr: One big repo doesn't actually cost that much, and that cost is dwarfed by the cost to humans of using submodules as proposed. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
On 7/19/16 8:55 AM, Robinson, Paul wrote: I think we could emulate any pre-commit hook we like via GitHub WebHooks by having two repositories: llvm and llvm-staging (say). People push to llvm-staging, which notifies some LLVM server we own. That does basic sanity checks and pushes to llvm proper if passed. I think that would be terrible in practice, for instance how do you handle situations like: 1) Dev A push commit A 2) Dev B push commit B that changes some lines close to the one changed by commit A 3) sanity check fails on commit A, but llvm-staging contains A and B and can’t get rid of A easily because B would not apply without A. At this point Dev B gets an email (or other mechanism, I don’t know what you imagined) telling that his changed are rejected for no good reason. Also reverting to a state "before A” on llvm-staging would mean that that the history would be rewritten and everyone that pulled/fetched in the meantime would suffer . If we want to go towards pre-check using staging, I believe we should work with pull-request (we’d still have the issue of conflicting PR, but I don’t believe it’d be that bad in practice). That’d be welcome, but that’d also be a whole other story to setup and maintain! — Mehdi Hear hear. The schemes to handle this that I'm aware of do look more like pull requests. You submit your change to the pre-qualification queue. If it rebases cleanly and passes pre-qual, your change becomes the new HEAD. If it doesn't rebase cleanly or fails pre-qual, your change gets bounced. Reminds me a bit of a blockchain: longest validated chain of commits wins. Jon A pull-request-like model also helps avoid the rebase-build-test-push(fail) loop that you can get into with a very active git-based project. This is a mechanical task best suited to automation rather than wasting valuable developer time on it. But I suspect GitHub does not have anything like this OOB so it would be an enhancement for a later time. --paulr P.S. At Sony we are building a system with a "staging" step but it's not for our own work... more of a "flood control" dam. :-) -- Jon Roelofs jonat...@codesourcery.com CodeSourcery / Mentor Embedded ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
> > I think we could emulate any pre-commit hook we like via GitHub > > WebHooks by having two repositories: llvm and llvm-staging (say). > > > > People push to llvm-staging, which notifies some LLVM server we own. > > That does basic sanity checks and pushes to llvm proper if passed. > > I think that would be terrible in practice, for instance how do you handle > situations like: > > 1) Dev A push commit A > 2) Dev B push commit B that changes some lines close to the one changed by > commit A > 3) sanity check fails on commit A, but llvm-staging contains A and B and > can’t get rid of A easily because B would not apply without A. > > At this point Dev B gets an email (or other mechanism, I don’t know what > you imagined) telling that his changed are rejected for no good reason. > > Also reverting to a state "before A” on llvm-staging would mean that that > the history would be rewritten and everyone that pulled/fetched in the > meantime would suffer . > > If we want to go towards pre-check using staging, I believe we should work > with pull-request (we’d still have the issue of conflicting PR, but I > don’t believe it’d be that bad in practice). > That’d be welcome, but that’d also be a whole other story to setup and > maintain! > > — > Mehdi Hear hear. The schemes to handle this that I'm aware of do look more like pull requests. You submit your change to the pre-qualification queue. If it rebases cleanly and passes pre-qual, your change becomes the new HEAD. If it doesn't rebase cleanly or fails pre-qual, your change gets bounced. A pull-request-like model also helps avoid the rebase-build-test-push(fail) loop that you can get into with a very active git-based project. This is a mechanical task best suited to automation rather than wasting valuable developer time on it. But I suspect GitHub does not have anything like this OOB so it would be an enhancement for a later time. --paulr P.S. At Sony we are building a system with a "staging" step but it's not for our own work... more of a "flood control" dam. :-) ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin updated this revision to Diff 64469. https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,254 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from our own +hosted Subversion to GitHub. Below are the financial and technical arguments as +to why we need such a move and how will people (and validation infrastructure) +continue to work with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + +* Outline of the reasons to move to Git and GitHub +* Description on what the work flow will look like (compared to SVN) +* Remaining issues and potential problems +* The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *free*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + +* Commit, squash, merge, fork locally without any penalty to the server +* Add as many branches as necessary to allow for multiple threads of development +* Collaborate with peers directly, even without access to the Internet +* Have multiple trees without multiplying disk space. + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide free code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and equality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people that still have/want to use SVN infrastructure and tooling can +slowly migrate or even stay working as if it was an SVN repository (including +read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the migration plan as +well as the community already settled there. + + +What will the new workflow look like + + +In order to move version control, we need to make sure that
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin updated this revision to Diff 64468. rengolin added a comment. Formatting issues (bullet points) https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,253 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from our own +hosted Subversion to GitHub. Below are the financial and technical arguments as +to why we need such a move and how will people (and validation infrastructure) +continue to work with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + +* Outline of the reasons to move to Git and GitHub +* Description on what the work flow will look like (compared to SVN) +* Remaining issues and potential problems +* The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *free*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: +* Commit, squash, merge, fork locally without any penalty to the server +* Add as many branches as necessary to allow for multiple threads of development +* Collaborate with peers directly, even without access to the Internet +* Have multiple trees without multiplying disk space. + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide free code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and equality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people that still have/want to use SVN infrastructure and tooling can +slowly migrate or even stay working as if it was an SVN repository (including +read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the migration plan as +well as the community already settled there. + + +What will the new workflow look like + + +
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin updated this revision to Diff 64467. rengolin added a comment. More updates, following recent comments. https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,253 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from our own +hosted Subversion to GitHub. Below are the financial and technical arguments as +to why we need such a move and how will people (and validation infrastructure) +continue to work with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + + * Outline of the reasons to move to Git and GitHub + * Description on how the work flow will look like (compared to SVN) + * Remaining issues and potential problems + * The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *free*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + * Commit, squash, merge, fork locally without any penalty to the server + * Add as many branches as necessary to allow for multiple threads of development + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space. + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide free code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and equality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people that still have/want to use SVN infrastructure and tooling can +slowly migrate or even stay working as if it was an SVN repository (including +read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the migration plan as +well as the community already settled there. + + +What will the new workflow look like +==
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:68 @@ +67,3 @@ + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + vsk wrote: > What do you mean by multiple concurrent builds? With git worktree you can work on source code and build different things at the same time, but I guess this is a specific use case which is only made "easier" in git. I'll remove this extra comment. Comment at: docs/Proposals/GitHub.rst:78 @@ +77,3 @@ + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that dberris wrote: > nit: I see you use FREE in caps but this instance isn't *FREE* (as opposed to > the first mention above) -- consider making it consistent? Either remove the > emphasis (just "free") or emphasise consistently? good point. Comment at: docs/Proposals/GitHub.rst:86 @@ +85,3 @@ +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + dberris wrote: > Did you mean "diversity and equality" instead of "diversity and quality" here? indeed, fixed. Comment at: docs/Proposals/GitHub.rst:102 @@ +101,3 @@ + +How will the new workflow look like +=== delcypher wrote: > s/How will/What will/ ok Comment at: docs/Proposals/GitHub.rst:110-112 @@ +109,5 @@ + +As with the current SVN and Git repositories, each project will be separate, +on its own, and people will also be able to check them out in the same way +they're already doing today. + dberris wrote: > Consider rewording this sentence -- it's a little too long and is trying to > say too many things. > > Perhaps something like: > > "Each LLVM project will continue to be hosted as separate GitHub repositories > under a single GitHub organisation. Users can continue to choose to use > either SVN or Git to access the repositories to suit their current workflow." Much better, thanks! Comment at: docs/Proposals/GitHub.rst:136 @@ +135,3 @@ + +We will need additional server hooks to avoid non-fast-forwards commits (ex. +merges, forced pushes, etc) in order to keep the linearity of the history. delcypher wrote: > @rengolin : I know GitHub enterprise has a "protected branch" feature to > prevent forced pushed ( > https://github.com/blog/2051-protected-branches-and-required-status-checks ). > You might want to speak to them to see if they can offer us that feature. I > think there's a limited support to do a merge as a squash and rebase too ( > https://github.com/blog/2141-squash-your-commits ) I'm asking about protected branches (it was proposed earlier, but I can't find any info on it). But we don't want to squash people's commits. They can do that on their own before commit. Comment at: docs/Proposals/GitHub.rst:185 @@ +184,3 @@ + +Are there any other upstream systems that could be affected? + dberris wrote: > vsk wrote: > > Yes, the `llvmlab bisect` functionality is affected. IMO it is invaluable > > for bug triage. Could you add some kind of reassurance that initially, > > updating it for the new VC model will just require pointing it to github's > > SVN view? > Probably worth mentioning how Phabricator will need to be updated to > integrate with the GitHub repository once the canonical repo is changed. Adding llvmlab bisect and Phab to the list. Both should be trivial. Comment at: docs/Proposals/GitHub.rst:209 @@ +208,3 @@ + well as a webhook to update the umbrella project (see below). +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). jyknight wrote: > mehdi_amini wrote: > > rengolin wrote: > > > mehdi_amini wrote: > > > > > Pre-commit hooks > > > > > > > > Won't handle the update of the umbrella. > > > > > > > > > Post-commit hooks > > > > > > > > Can't handle the update of the umbrella *because of GitHub*, this could > > > > be possible with our own hosting of git for instance. > > > > > > > Pre-commit hooks are not designed to update the umbrella. Webhooks will > > > be able to update the umbrella with a small external service, as proposed > > > in the IRC. > > That's why I asked it to be clearly mentioned somewhere else that on top of > > "hooks" hosted by github, we will need to maintain our own webservice on > > our own server to answer updates from theses hooks and update the umbrella, > > because that's a non-negligible drawback in face of the motivation exposed > > in the "Why move at all?" section. > > Right now the document doe
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
filcab added a comment. Thanks a lot for working on this! Filipe https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
> On Jul 18, 2016, at 8:23 PM, Tim Northover via cfe-commits > wrote: > >>> Can't handle the update of the umbrella *because of GitHub*, this could be >>> possible with our own hosting of git for instance. >>> >> Pre-commit hooks are not designed to update the umbrella. Webhooks will be >> able to update the umbrella with a small external service, as proposed in >> the IRC. > > I think we could emulate any pre-commit hook we like via GitHub > WebHooks by having two repositories: llvm and llvm-staging (say). > > People push to llvm-staging, which notifies some LLVM server we own. > That does basic sanity checks and pushes to llvm proper if passed. I think that would be terrible in practice, for instance how do you handle situations like: 1) Dev A push commit A 2) Dev B push commit B that changes some lines close to the one changed by commit A 3) sanity check fails on commit A, but llvm-staging contains A and B and can’t get rid of A easily because B would not apply without A. At this point Dev B gets an email (or other mechanism, I don’t know what you imagined) telling that his changed are rejected for no good reason. Also reverting to a state "before A” on llvm-staging would mean that that the history would be rewritten and everyone that pulled/fetched in the meantime would suffer . If we want to go towards pre-check using staging, I believe we should work with pull-request (we’d still have the issue of conflicting PR, but I don’t believe it’d be that bad in practice). That’d be welcome, but that’d also be a whole other story to setup and maintain! — Mehdi > > It has disadvantages (no instant "success" feedback being the obvious > one), but would allow us to vet commits with relatively little > overhead (as James says, running a server responding to webhooks is a > very different proposition from one hammered by hundreds of developers > daily). > > I'm not strongly in favour of this, just thought I'd mention it as a > possibility. > > Tim. > ___ > cfe-commits mailing list > cfe-comm...@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
dberris added a subscriber: dberris. dberris added a comment. Mostly wording comments, thank you for writing this up! Comment at: docs/Proposals/GitHub.rst:78 @@ +77,3 @@ + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that nit: I see you use FREE in caps but this instance isn't *FREE* (as opposed to the first mention above) -- consider making it consistent? Either remove the emphasis (just "free") or emphasise consistently? Comment at: docs/Proposals/GitHub.rst:86 @@ +85,3 @@ +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + Did you mean "diversity and equality" instead of "diversity and quality" here? Comment at: docs/Proposals/GitHub.rst:110-112 @@ +109,5 @@ + +As with the current SVN and Git repositories, each project will be separate, +on its own, and people will also be able to check them out in the same way +they're already doing today. + Consider rewording this sentence -- it's a little too long and is trying to say too many things. Perhaps something like: "Each LLVM project will continue to be hosted as separate GitHub repositories under a single GitHub organisation. Users can continue to choose to use either SVN or Git to access the repositories to suit their current workflow." Comment at: docs/Proposals/GitHub.rst:185 @@ +184,3 @@ + +Are there any other upstream systems that could be affected? + Probably worth mentioning how Phabricator will need to be updated to integrate with the GitHub repository once the canonical repo is changed. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
friss added inline comments. Comment at: docs/Proposals/GitHub.rst:220 @@ +219,3 @@ +8. Tell people living downstream to pick up commits from the official git + repository. +9. Give things time to settle. We could play some games like disabling the SVN vsk wrote: > This is tricky. CC'ing @friss to comment on how we'd need to update our > internal auto-merger. This is not an issue. We were already consuming the llvm.org git repos, so we'd just need to point to the new git repos (given the git history is preserved, I don't think I've read this anywhere in this document, but I think it's safe to make this assumption) https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
>> Can't handle the update of the umbrella *because of GitHub*, this could be >> possible with our own hosting of git for instance. >> > Pre-commit hooks are not designed to update the umbrella. Webhooks will be > able to update the umbrella with a small external service, as proposed in the > IRC. I think we could emulate any pre-commit hook we like via GitHub WebHooks by having two repositories: llvm and llvm-staging (say). People push to llvm-staging, which notifies some LLVM server we own. That does basic sanity checks and pushes to llvm proper if passed. It has disadvantages (no instant "success" feedback being the obvious one), but would allow us to vet commits with relatively little overhead (as James says, running a server responding to webhooks is a very different proposition from one hammered by hundreds of developers daily). I'm not strongly in favour of this, just thought I'd mention it as a possibility. Tim. ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
vsk added subscribers: friss, vsk. vsk added a comment. @rengolin thanks for putting this together! I chimed in with some comments in-line. Comment at: docs/Proposals/GitHub.rst:68 @@ +67,3 @@ + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + What do you mean by multiple concurrent builds? Comment at: docs/Proposals/GitHub.rst:185 @@ +184,3 @@ + +Are there any other upstream systems that could be affected? + Yes, the `llvmlab bisect` functionality is affected. IMO it is invaluable for bug triage. Could you add some kind of reassurance that initially, updating it for the new VC model will just require pointing it to github's SVN view? Comment at: docs/Proposals/GitHub.rst:220 @@ +219,3 @@ +8. Tell people living downstream to pick up commits from the official git + repository. +9. Give things time to settle. We could play some games like disabling the SVN This is tricky. CC'ing @friss to comment on how we'd need to update our internal auto-merger. Comment at: docs/Proposals/GitHub.rst:233 @@ +232,3 @@ + +10. Collect peoples GitHub account information, adding them to the project. +11. Switch SVN repository to read-only and allow pushes to the GitHub repository. delcypher wrote: > GitHub organizations support the notion of teams which can each have > different permissions (for example you'll want to make sure only the right > people have admin access and give the rest write/read access). You could also > make a team per project so that write access in one project does not give > write access to another. I think it would be good to decide on how teams will > be organized and state this in the document. I think this is an important discussion to have once the move is under-way. I don't think finer-grained write privileges should be a part of this proposal since it's (1) a separate issue and (2) not *just* an artifact of llvm-project's svn structure (i.e there are good reasons to keep the current permissions model in place). https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jroelofs added a subscriber: jroelofs. Comment at: docs/Proposals/GitHub.rst:127 @@ +126,3 @@ +* The projects' repositories will remain identical, with a new address (GitHub). +* They'll continue to have SVN RW access, but will also gain Git RW access. +* The linear history can still be accessed in the (RO) submodule meta project, rengolin wrote: > compnerd wrote: > > jroelofs wrote: > > > Do you mean `s/SVN RW access/SVN RO access/` here? > > I believe @rengolin is referring to the final state here. I agree that the > > current phrasing makes it hard to follow. > No, I actually mean SVN RW access. GitHub's SVN view does allow write access > to the Git repos via "svn commit". Ah, I didn't catch that part. Cool. Comment at: docs/Proposals/GitHub.rst:136 @@ +135,3 @@ +Essentially, we're adding Git RW access in addition to the already existing +structure, with all the additional benefits of it being in GitHub. + rengolin wrote: > compnerd wrote: > > jroelofs wrote: > > > Need to clarify here whether *write* access through SVN will be going > > > away. If I understand the proposal correctly, it will go away, but this > > > section makes it sound like it's staying. > > The way that I read the nutshell is that it would potentially continue to > > exist, just at a different address. > Our SVN server will die, SVN access will continue via GitHub. Ah, ok. Repository: rL LLVM https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
probinson added a subscriber: probinson. Comment at: docs/Proposals/GitHub.rst:141 @@ +140,3 @@ +has commit access to our current repository. In the future, you only need to +provide the GitHub user to be granted access. + This reads a little bit like "we will create a GitHub account for you if you don't have one" but I suspect people actually need to create their own GitHub accounts first. (We're not all on GitHub already!) https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
compnerd added inline comments. Comment at: docs/Proposals/GitHub.rst:127 @@ +126,3 @@ +* The projects' repositories will remain identical, with a new address (GitHub). +* They'll continue to have SVN RW access, but will also gain Git RW access. +* The linear history can still be accessed in the (RO) submodule meta project, jroelofs wrote: > Do you mean `s/SVN RW access/SVN RO access/` here? I believe @rengolin is referring to the final state here. I agree that the current phrasing makes it hard to follow. Comment at: docs/Proposals/GitHub.rst:129 @@ +128,3 @@ +* The linear history can still be accessed in the (RO) submodule meta project, + Which will continue to have SVN access. + "Which will continue to have SVN access" is redundant given the previous statement. Comment at: docs/Proposals/GitHub.rst:136 @@ +135,3 @@ +Essentially, we're adding Git RW access in addition to the already existing +structure, with all the additional benefits of it being in GitHub. + jroelofs wrote: > Need to clarify here whether *write* access through SVN will be going away. > If I understand the proposal correctly, it will go away, but this section > makes it sound like it's staying. The way that I read the nutshell is that it would potentially continue to exist, just at a different address. Comment at: docs/Proposals/GitHub.rst:155 @@ +154,3 @@ +But some people/companies might not be allowed to use GitHub or might have +firewalls blocking certain websites. + GitHub does have HTTPS based connections. It seems highly unlikely that this is a real concern. Companies would have to go out of their way to block access specifically to github over SSH and HTTPS. Comment at: docs/Proposals/GitHub.rst:167 @@ +166,3 @@ +with the limited number of developers whose job will be to mainly merge +thousands of patches a day. + I don't fully understand how this is any different from today. We have a core set of developers with commit access. Others are encouraged to provide patches via email (or may use phabricator depending on the reviewer). Once reviewed and accepted, one of the core developers still commits the change. I just see this as a process change. The person forks the repository on github, and creates a branch, and then a PR. The PR is reviewed and once accepted, merged by one of the core developers. It even implicitly handles authorship tracking which has currently been done in an adhoc fashion via the commit message. Comment at: docs/Proposals/GitHub.rst:222 @@ +221,3 @@ +10. Collect peoples GitHub account information, give them push access. Ideally +while still locking the GitHub repository somehow... +11. Switch SVN repository to read-only and allow pushes to the GitHub repository. Giving permissions to only the LLVM "project" is sufficient. People can be added to the LLVM "project" as collaborators and get access that way. This is similar to how Apple is managing swift for comparison. Repository: rL LLVM https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
mehdi_amini added a subscriber: mehdi_amini. Comment at: docs/Proposals/GitHub.rst:122 @@ +121,3 @@ +of understanding the *sequence* in which commits were added by using the +``git rev-list --count hash`` or ``git describe hash`` commands. + filcab wrote: > How easy will it be to clone the "aggregated" repo, and then get some (but > not all) of the submodules? I expect the umbrella repo to come with scripts for that. Comment at: docs/Proposals/GitHub.rst:129 @@ +128,3 @@ +* The linear history can still be accessed in the (RO) submodule meta project. +* Individual projects' history will be broken (linear, but local), and we need + the umbrella project (using submodules) to have the same view as we had in SVN. This sentence is not clear: "Individual projects' history will be broken", I don't see what's "broken". Comment at: docs/Proposals/GitHub.rst:168 @@ +167,3 @@ +additional benefits. + +2. Which tools will stop working? I think that's covered line 136-137. Comment at: docs/Proposals/GitHub.rst:173 @@ +172,3 @@ +use LNT with the SVN-View, but it would be best to move it to Git once and for +all. + I don't think so: LNT is not dependent on SVN history. It is dependent on a single revision number cross-repository and cross-branches. This is something that the umbrella projects "could" provide. Comment at: docs/Proposals/GitHub.rst:198 @@ +197,3 @@ +3. Make sure we have an llvm-project (with submodules) setup in the official + account. +4. Make sure bisecting with llvm-project works. Uh, this point is not clear: there will a need for us to maintain a "non-trivial" hook on our server to handle this. This is not fleshed out in this document. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
mehdi_amini added inline comments. Comment at: docs/Proposals/GitHub.rst:209 @@ +208,3 @@ + well as a webhook to update the umbrella project (see below). +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). > Pre-commit hooks Won't handle the update of the umbrella. > Post-commit hooks Can't handle the update of the umbrella *because of GitHub*, this could be possible with our own hosting of git for instance. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
mehdi_amini added inline comments. Comment at: docs/Proposals/GitHub.rst:199 @@ +198,3 @@ + +Here's a proposed plan: + Annoyingly my comment does no longer show-up next to the point it was referring to, it was about your third point: > Make sure we have an llvm-project (with submodules) setup in the official > account. I don't think how this project will be updated is explained (or even mentioned) anywhere. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
winksaville added a subscriber: winksaville. Comment at: docs/Proposals/GitHub.rst:132 @@ +131,3 @@ + +There is no need to additional tags, flags and properties, nor of external +services controlling the history, since both SVN and *git rev-list* can already This could be worded a little better, may I suggest something like: "There is no need for additional tags, flags, properties, or external ..." https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:127 @@ +126,3 @@ +* The projects' repositories will remain identical, with a new address (GitHub). +* They'll continue to have SVN RW access, but will also gain Git RW access. +* The linear history can still be accessed in the (RO) submodule meta project, jroelofs wrote: > Do you mean `s/SVN RW access/SVN RO access/` here? No, I actually mean SVN RW access. GitHub's SVN view does allow write access to the Git repos via "svn commit". Comment at: docs/Proposals/GitHub.rst:136 @@ +135,3 @@ +Essentially, we're adding Git RW access in addition to the already existing +structure, with all the additional benefits of it being in GitHub. + jroelofs wrote: > Need to clarify here whether *write* access through SVN will be going away. > If I understand the proposal correctly, it will go away, but this section > makes it sound like it's staying. Our SVN server will die, SVN access will continue via GitHub. Repository: rL LLVM https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
delcypher added a subscriber: delcypher. Comment at: docs/Proposals/GitHub.rst:102 @@ +101,3 @@ + +How will the new workflow look like +=== s/How will/What will/ Comment at: docs/Proposals/GitHub.rst:136 @@ +135,3 @@ + +We will need additional server hooks to avoid non-fast-forwards commits (ex. +merges, forced pushes, etc) in order to keep the linearity of the history. @rengolin : I know GitHub enterprise has a "protected branch" feature to prevent forced pushed ( https://github.com/blog/2051-protected-branches-and-required-status-checks ). You might want to speak to them to see if they can offer us that feature. I think there's a limited support to do a merge as a squash and rebase too ( https://github.com/blog/2141-squash-your-commits ) Comment at: docs/Proposals/GitHub.rst:233 @@ +232,3 @@ + +10. Collect peoples GitHub account information, adding them to the project. +11. Switch SVN repository to read-only and allow pushes to the GitHub repository. GitHub organizations support the notion of teams which can each have different permissions (for example you'll want to make sure only the right people have admin access and give the rest write/read access). You could also make a team per project so that write access in one project does not give write access to another. I think it would be good to decide on how teams will be organized and state this in the document. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
mehdi_amini added inline comments. Comment at: docs/Proposals/GitHub.rst:209 @@ +208,3 @@ + well as a webhook to update the umbrella project (see below). +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). rengolin wrote: > mehdi_amini wrote: > > > Pre-commit hooks > > > > Won't handle the update of the umbrella. > > > > > Post-commit hooks > > > > Can't handle the update of the umbrella *because of GitHub*, this could be > > possible with our own hosting of git for instance. > > > Pre-commit hooks are not designed to update the umbrella. Webhooks will be > able to update the umbrella with a small external service, as proposed in the > IRC. That's why I asked it to be clearly mentioned somewhere else that on top of "hooks" hosted by github, we will need to maintain our own webservice on our own server to answer updates from theses hooks and update the umbrella, because that's a non-negligible drawback in face of the motivation exposed in the "Why move at all?" section. Right now the document does not acknowledge that AFAICT. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
mehdi_amini added inline comments. Comment at: docs/Proposals/GitHub.rst:208 @@ +207,3 @@ +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). +4. Make sure bisecting with llvm-project works. I'd like to see clearly mentioned somewhere else than in the plan that on top of "hooks" hosted by github, we will need to maintain our own webservice on our own server to answer updates from theses hooks and update the umbrella. That's a non-negligible drawback in face of the motivation exposed in the "Why move at all?" section. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jroelofs added inline comments. Comment at: docs/Proposals/GitHub.rst:127 @@ +126,3 @@ +* The projects' repositories will remain identical, with a new address (GitHub). +* They'll continue to have SVN RW access, but will also gain Git RW access. +* The linear history can still be accessed in the (RO) submodule meta project, Do you mean `s/SVN RW access/SVN RO access/` here? Comment at: docs/Proposals/GitHub.rst:136 @@ +135,3 @@ +Essentially, we're adding Git RW access in addition to the already existing +structure, with all the additional benefits of it being in GitHub. + Need to clarify here whether *write* access through SVN will be going away. If I understand the proposal correctly, it will go away, but this section makes it sound like it's staying. Repository: rL LLVM https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
jyknight added a subscriber: jyknight. Comment at: docs/Proposals/GitHub.rst:209 @@ +208,3 @@ + well as a webhook to update the umbrella project (see below). +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). mehdi_amini wrote: > rengolin wrote: > > mehdi_amini wrote: > > > > Pre-commit hooks > > > > > > Won't handle the update of the umbrella. > > > > > > > Post-commit hooks > > > > > > Can't handle the update of the umbrella *because of GitHub*, this could > > > be possible with our own hosting of git for instance. > > > > > Pre-commit hooks are not designed to update the umbrella. Webhooks will be > > able to update the umbrella with a small external service, as proposed in > > the IRC. > That's why I asked it to be clearly mentioned somewhere else that on top of > "hooks" hosted by github, we will need to maintain our own webservice on our > own server to answer updates from theses hooks and update the umbrella, > because that's a non-negligible drawback in face of the motivation exposed in > the "Why move at all?" section. > Right now the document does not acknowledge that AFAICT. The maintenance of that service will be negligible compared to running a subversion installation. I expect that we could set up the webhook as an AppEngine app, using Github's "Git Data" API to generate the new commit, and then to the first approximation never have to touch it again. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:209 @@ +208,3 @@ + well as a webhook to update the umbrella project (see below). +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). mehdi_amini wrote: > > Pre-commit hooks > > Won't handle the update of the umbrella. > > > Post-commit hooks > > Can't handle the update of the umbrella *because of GitHub*, this could be > possible with our own hosting of git for instance. > Pre-commit hooks are not designed to update the umbrella. Webhooks will be able to update the umbrella with a small external service, as proposed in the IRC. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin updated this revision to Diff 64383. rengolin added a comment. Expand step 2 to make sure we don't forget about the safety hooks on each project as well as the webhook to update the umbrella project. This could turn out to be a buildbot, but makes no difference at this stage. https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,242 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from our own +hosted Subversion to GitHub. Below are the financial and technical arguments as +to why we need such a move and how will people (and validation infrastructure) +continue to work with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + + * Outline of the reasons to move to Git and GitHub + * Description on how the work flow will look like (compared to SVN) + * Remaining issues and potential problems + * The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *FREE*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + * Commit, squash, merge, fork locally without any penalty to the server + * Add as many branches as necessary to allow for multiple threads of development + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people that still have/want to use SVN infrastructure and tooling can +slowly migrate or even stay working as if it was an SVN repository (including +read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:208 @@ +207,3 @@ +3. Make sure we have an llvm-project (with submodules) setup in the official + account, with all necessary hooks (history, update, merges). +4. Make sure bisecting with llvm-project works. mehdi_amini wrote: > I'd like to see clearly mentioned somewhere else than in the plan that on top > of "hooks" hosted by github, we will need to maintain our own webservice on > our own server to answer updates from theses hooks and update the umbrella. > That's a non-negligible drawback in face of the motivation exposed in the > "Why move at all?" section. There are two types of hooks: 1. Pre-commit hooks that will stop anyone trying to merge/force push commits to the projects, in order to keep the history clean. These are install once, use forever. Zero maintenance after the initial period. 2. Post-commit hooks on the other projects / OR / external webservice/buildbot that will update the umbrella project like any existing Git mirror. Maintenance of this is either free (hooks) or very cheap (buildbot/cron jobs). On both cases, the history is preserved at least within the update cycle, which shouldn't be more than 5 minutes, and will be the update cycle that buildbots will pick the commits anyway. https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:200 @@ +199,3 @@ + +Here's a proposed plan: + You can click on the "<<" button and it will show where it was first inserted. That's how I found out. :) The hooks, AFAICS, will be added to the project initially, and won't need to be updated. Takumi's current repository is automatically updated, and IIRC, it's just a hook. Assuming it's atomic and quick enough, would this process make much of a difference to the users? https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin updated this revision to Diff 64373. rengolin added a comment. Removing "broken" to describe the history, just explaining it'll be local. Expanding to mention that hooks will need to be implemented in step 3. https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,240 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from our own +hosted Subversion to GitHub. Below are the financial and technical arguments as +to why we need such a move and how will people (and validation infrastructure) +continue to work with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + + * Outline of the reasons to move to Git and GitHub + * Description on how the work flow will look like (compared to SVN) + * Remaining issues and potential problems + * The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *FREE*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + * Commit, squash, merge, fork locally without any penalty to the server + * Add as many branches as necessary to allow for multiple threads of development + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people that still have/want to use SVN infrastructure and tooling can +slowly migrate or even stay working as if it was an SVN repository (including +read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the m
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:198 @@ +197,3 @@ +3. Make sure we have an llvm-project (with submodules) setup in the official + account. +4. Make sure bisecting with llvm-project works. mehdi_amini wrote: > Uh, this point is not clear: there will a need for us to maintain a > "non-trivial" hook on our server to handle this. This is not fleshed out in > this document. Good point. I added just a description to this topic, since this is covered elsewhere. Comment at: docs/Proposals/GitHub.rst:174 @@ +173,3 @@ +us what's wrong and how to help them fix it. + +We also recommend people and companies to migrate to Git, for its many other So, LNT migration plan could be: 1. Use GitHub's SVN view on llvm-proj-submods 2. Move to understand submods 3. Migrate all instances Looks fairly orthogonal to me... https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin updated this revision to Diff 64371. rengolin added a comment. Second round of suggestions applied. https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,239 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from our own +hosted Subversion to GitHub. Below are the financial and technical arguments as +to why we need such a move and how will people (and validation infrastructure) +continue to work with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + + * Outline of the reasons to move to Git and GitHub + * Description on how the work flow will look like (compared to SVN) + * Remaining issues and potential problems + * The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *FREE*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + * Commit, squash, merge, fork locally without any penalty to the server + * Add as many branches as necessary to allow for multiple threads of development + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people that still have/want to use SVN infrastructure and tooling can +slowly migrate or even stay working as if it was an SVN repository (including +read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the migration plan as +well as the community already settled there. + + +How will the new workflow look like +=
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:8-9 @@ +7,4 @@ + +This is a proposal to move our current revision control system from Subversion +to GitHub. Below are the financial and technical arguments as to why we need +such a move and how will people (and validation infrastructure) continue to work emaste wrote: > It seems pedantic, but I think we should try hard to avoid conflating Git and > GitHub. What about: "move our revision control system from //self-hosted// > Subversion to Git hosted by GitHub." This is a summary of the whole proposal, which specifically dictates GitHub. We're not proposing to move to some Git hosting anymore, but exactly GitHub, due to the constraints that we outline below. If we do not move to GitHub specifically, a lot of the assumptions below will be wrong, and this proposal can't be accepted. There is a paragraph on why git, and another on why GitHub, and both are key points of this proposal. I'll change "from Subversion" to "from our own hosted Subversion" to make that even more clear. Comment at: docs/Proposals/GitHub.rst:93 @@ +92,3 @@ +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people stuck with SVN infrastructure and tooling can slowly migrate or +even stay working as if it was an SVN repository (including read-write access). emaste wrote: > Replace "stuck" with something neutral. "stuck" implies that everyone will > want to move but some may not be able to for technical or other reasons, but > some people actually prefer SVN. good point. Comment at: docs/Proposals/GitHub.rst:122 @@ +121,3 @@ +of understanding the *sequence* in which commits were added by using the +``git rev-list --count hash`` or ``git describe hash`` commands. + filcab wrote: > How easy will it be to clone the "aggregated" repo, and then get some (but > not all) of the submodules? Checking out this project: https://github.com/chapuni/llvm-project-submodule Will return the references, not the sub modules. You have to "init" each sub-module independently, which achieves the same task as only checking out the SVN repos you need, with the correct numbering. Comment at: docs/Proposals/GitHub.rst:130 @@ +129,3 @@ +* Individual projects' history will be broken (linear, but local), and we need + the umbrella project (using submodules) to have the same view as we had in SVN. + filcab wrote: > I wouldn't call it broken. > Won't it have the same end result as having a checkout per project and simply > updating them close to each other? > > Basically, it won't be "any more broken" than using this method for updating: > > ``` > #!/bin/bash > for dir in llvm/{,tools/{clang,lld},projects/{libcxx,libcxxabi,compiler-rt}}; > do > # (cd $dir && svn up) # for SVN > (cd $dir && git checkout master && git pull) # for git > done > ``` No, it won't. Checking out LLVM only skips all commits from all other repos. So, for example: LNT 123 Clang 124 RT 125 LLVM 126 Then, "svn checkout 126" will be: In LNT, 123 as 126 In Clang, 124 as 126 In RT, 125 as 126 In LLVM, 126 as 126 With the new SVN interface, each one of those commits will be referred to, locally, as 123, and 126 will not exist, because the "git rev-list --count" won't get as high as 126. However, on the umbrella submodule project, because the sequence of the commits is guaranteed, the rev-list count will bring the correct numbering, the same as via the SVN interface, and thus be "just like SVN was". Comment at: docs/Proposals/GitHub.rst:132 @@ +131,3 @@ + +There is no need to additional tags, flags and properties, nor of external +services controlling the history, since both SVN and *git rev-list* can already winksaville wrote: > This could be worded a little better, may I suggest something like: > > "There is no need for additional tags, flags, properties, or external ..." > Yup, changing on next round. Thanks! Comment at: docs/Proposals/GitHub.rst:141 @@ +140,3 @@ +has commit access to our current repository. In the future, you only need to +provide the GitHub user to be granted access. + probinson wrote: > This reads a little bit like "we will create a GitHub account for you if you > don't have one" but I suspect people actually need to create their own GitHub > accounts first. (We're not all on GitHub already!) well, "you need to provide the GitHub user" should take care of that, but I'll try to re-write this to make it more clear. > (We're not all on GitHub already!) Are we not? Egregious! :D Comment at: docs/Proposals/GitHub.rst:180 @@ +179,3 @@ + +As soon as we decide to move, we'll have to set a date for the process to begin. + emaste wrote: > This presents it
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
emaste added a subscriber: emaste. Comment at: docs/Proposals/GitHub.rst:8-9 @@ +7,4 @@ + +This is a proposal to move our current revision control system from Subversion +to GitHub. Below are the financial and technical arguments as to why we need +such a move and how will people (and validation infrastructure) continue to work It seems pedantic, but I think we should try hard to avoid conflating Git and GitHub. What about: "move our revision control system from //self-hosted// Subversion to Git hosted by GitHub." Comment at: docs/Proposals/GitHub.rst:93 @@ +92,3 @@ +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people stuck with SVN infrastructure and tooling can slowly migrate or +even stay working as if it was an SVN repository (including read-write access). Replace "stuck" with something neutral. "stuck" implies that everyone will want to move but some may not be able to for technical or other reasons, but some people actually prefer SVN. Comment at: docs/Proposals/GitHub.rst:180 @@ +179,3 @@ + +As soon as we decide to move, we'll have to set a date for the process to begin. + This presents it as if the decision is already made, which somewhat defeats the purpose of writing a proposal for the LLVM community to vote on. Maybe "If we decide to move"? https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
filcab added a subscriber: filcab. filcab added a comment. What about branches? I'm guessing we should expect the usual release branches. But will any person be able to create a branch? Will there be a policy, if this is the case? Is the policy enforceable? Comment at: docs/Proposals/GitHub.rst:122 @@ +121,3 @@ +of understanding the *sequence* in which commits were added by using the +``git rev-list --count hash`` or ``git describe hash`` commands. + How easy will it be to clone the "aggregated" repo, and then get some (but not all) of the submodules? Comment at: docs/Proposals/GitHub.rst:130 @@ +129,3 @@ +* Individual projects' history will be broken (linear, but local), and we need + the umbrella project (using submodules) to have the same view as we had in SVN. + I wouldn't call it broken. Won't it have the same end result as having a checkout per project and simply updating them close to each other? Basically, it won't be "any more broken" than using this method for updating: ``` #!/bin/bash for dir in llvm/{,tools/{clang,lld},projects/{libcxx,libcxxabi,compiler-rt}}; do # (cd $dir && svn up) # for SVN (cd $dir && git checkout master && git pull) # for git done ``` https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin removed rL LLVM as the repository for this revision. rengolin updated this revision to Diff 64334. rengolin added a comment. First round of changes reflecting reviews. https://reviews.llvm.org/D22463 Files: docs/Proposals/GitHub.rst Index: docs/Proposals/GitHub.rst === --- /dev/null +++ docs/Proposals/GitHub.rst @@ -0,0 +1,230 @@ +== +Moving LLVM Projects to GitHub +== + +Introduction + + +This is a proposal to move our current revision control system from Subversion +to GitHub. Below are the financial and technical arguments as to why we need +such a move and how will people (and validation infrastructure) continue to work +with a Git-based LLVM. + +There will be a survey pointing at this document when we'll know the community's +reaction and, if we collectively decide to move, the time-frames. Be sure to make +your views count. + +Essentially, the proposal is divided in the following parts: + + * Outline of the reasons to move to Git and GitHub + * Description on how the work flow will look like (compared to SVN) + * Remaining issues and potential problems + * The proposed migration plan + +Why Git, and Why GitHub? + + +Why move at all? + + +The strongest reason for the move, and why this discussion started in the first +place, is that we currently host our own Subversion server and Git mirror in a +voluntary basis. The LLVM Foundation sponsors the server and provides limited +support, but there is only so much it can do. + +The volunteers are not Sysadmins themselves, but compiler engineers that happen +to know a thing or two about hosting servers. We also don't have 24/7 support, +and we sometimes wake up to see that continuous integration is broken because +the SVN server is either down or unresponsive. + +With time and money, the foundation and volunteers could improve our services, +implement more functionality and provide around the clock support, so that we +can have a first class infrastructure with which to work. But the cost is not +small, both in money and time invested. + +On the other hand, there are multiple services out there (GitHub, GitLab, +BitBucket among others) that offer that same service (24/7 stability, disk space, +Git server, code browsing, forking facilities, etc) for the very affordable price +of *FREE*. + +Why Git? + + +Most new coders nowadays start with Git. A lot of them have never used SVN, CVS +or anything else. Websites like GitHub have changed the landscape of open source +contributions, reducing the cost of first contribution and fostering +collaboration. + +Git is also the version control most LLVM developers use. Despite the sources +being stored in an SVN server, most people develop using the Git-SVN integration, +and that shows that Git is not only more powerful than SVN, but people have +resorted to using a bridge because its features are now indispensable to their +internal and external workflows. + +In essence, Git allows you to: + * Commit, squash, merge, fork locally without any penalty to the server + * Add as many branches as necessary to allow for multiple threads of development + * Collaborate with peers directly, even without access to the Internet + * Have multiple trees without multiplying disk space, multiple concurrent builds + +In addition, because Git seems to be replacing every project's version control +system, there are many more tools that can use Git's enhanced feature set, so +new tooling is much more likely to support Git first (if not only), than any +other version control system. + +Why GitHub? +--- + +GitHub, like GitLab and BitBucket, provide FREE code hosting for open source +projects. Essentially, they will completely replace *all* the infrastructure that +we have today that serves code repository, mirroring, user control, etc. + +They also have a dedicated team to monitor, migrate, improve and distribute the +contents of the repositories depending on region and load. A level of quality +that we'd never have without spending money that would be better spent elsewhere, +for example development meetings, sponsoring disadvantaged people to work on +compilers and foster diversity and quality in our community. + +GitHub has the added benefit that we already have a presence there. Many +developers use it already, and the mirror from our current repository is already +set up. + +Furthermore, GitHub has an *SVN view* (https://github.com/blog/626-announcing-svn-support) +where people stuck with SVN infrastructure and tooling can slowly migrate or +even stay working as if it was an SVN repository (including read-write access). + +So, any of the three solutions solve the cost and maintenance problem, but GitHub +has two additional features that would be beneficial to the migration plan as +well as the community already settled there. + + +How wil
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
compnerd added inline comments. Comment at: docs/Proposals/GitHub.rst:167 @@ +166,3 @@ +with the limited number of developers whose job will be to mainly merge +thousands of patches a day. + rengolin wrote: > compnerd wrote: > > I don't fully understand how this is any different from today. We have a > > core set of developers with commit access. Others are encouraged to > > provide patches via email (or may use phabricator depending on the > > reviewer). Once reviewed and accepted, one of the core developers still > > commits the change. I just see this as a process change. > > > > The person forks the repository on github, and creates a branch, and then a > > PR. The PR is reviewed and once accepted, merged by one of the core > > developers. It even implicitly handles authorship tracking which has > > currently been done in an adhoc fashion via the commit message. > Today we all commit to SVN, which is linear. In GitHub, we'll be committing > to git. If we can have hooks forbidding merges, it'll remain linear, but then > pull requests will be blocked. Additional hooks will need to be in place > (please suggest all of them here and I'll update the doc). I think that we should aim to preserve the linearity of history. This would mean that we block non-fastforward commits (i.e. no merges, no force pushes). Repository: rL LLVM https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
Re: [Lldb-commits] [PATCH] D22463: [RFC] Moving to GitHub Proposal: NOT DECISION!
rengolin added inline comments. Comment at: docs/Proposals/GitHub.rst:129 @@ +128,3 @@ +* The linear history can still be accessed in the (RO) submodule meta project, + Which will continue to have SVN access. + compnerd wrote: > "Which will continue to have SVN access" is redundant given the previous > statement. good point. I'll try and re-write those points to be more clear. Comment at: docs/Proposals/GitHub.rst:155 @@ +154,3 @@ +But some people/companies might not be allowed to use GitHub or might have +firewalls blocking certain websites. + compnerd wrote: > GitHub does have HTTPS based connections. It seems highly unlikely that this > is a real concern. Companies would have to go out of their way to block > access specifically to github over SSH and HTTPS. I have had this problem in China... Though no one has raised this issue, so I'll just remove and let people complain about this in the survey. Comment at: docs/Proposals/GitHub.rst:167 @@ +166,3 @@ +with the limited number of developers whose job will be to mainly merge +thousands of patches a day. + compnerd wrote: > I don't fully understand how this is any different from today. We have a > core set of developers with commit access. Others are encouraged to provide > patches via email (or may use phabricator depending on the reviewer). Once > reviewed and accepted, one of the core developers still commits the change. > I just see this as a process change. > > The person forks the repository on github, and creates a branch, and then a > PR. The PR is reviewed and once accepted, merged by one of the core > developers. It even implicitly handles authorship tracking which has > currently been done in an adhoc fashion via the commit message. Today we all commit to SVN, which is linear. In GitHub, we'll be committing to git. If we can have hooks forbidding merges, it'll remain linear, but then pull requests will be blocked. Additional hooks will need to be in place (please suggest all of them here and I'll update the doc). Comment at: docs/Proposals/GitHub.rst:222 @@ +221,3 @@ +10. Collect peoples GitHub account information, give them push access. Ideally +while still locking the GitHub repository somehow... +11. Switch SVN repository to read-only and allow pushes to the GitHub repository. compnerd wrote: > Giving permissions to only the LLVM "project" is sufficient. People can be > added to the LLVM "project" as collaborators and get access that way. This > is similar to how Apple is managing swift for comparison. That's what I meant but I will change the wording. Repository: rL LLVM https://reviews.llvm.org/D22463 ___ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits