Re: Fwd: ghc-7.6 branch
On Thu, Jun 28, 2012 at 10:47 AM, Simon Peyton-Jones wrote: > | > Has maintainer's not being responsive been a problem for GHC in the > | > past? > | > | Yes. Some of the upstreams respond so fast that it makes my head spin, > | while others often either don't respond or continually promise to get to > | things soon. (again, these are good, well-meaning people, who do a lot > | for the community). > > The obvious solution, as someone else pointed out, is for someone at GHC HQ > (perhaps Ian) to be a co-maintainer of these critical dependencies -- except > perhaps for libraries whose maintainers are the "make your head spin" > responsive kind. This would of course need the maintainer to trust the GHC > person to push patches and make releases in sync with GHC. But my guess is > they'd be willing. After all, we're all on the same side here! That works for me. If maintainers want to delegate to GHC HQ (and GHC HQ has the bandwidth to deal with these releases) that is of course fine. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
Hi Ian, On Thu, Jun 28, 2012 at 6:26 AM, Ian Lynagh wrote: > You didn't give a clear answer to my question. Am I right in thinking > that your answer would be "Yes, the GHC release should be delayed > indefinitely"? I did answer it, just not with a "yes" or "no" as it's a false dichotomy. I gave you the 4 options that I think are reasonable. Perhaps someone could think of others, but those are the standard ones typically used by open source communities. > (or at least, for long enough for the maintainer to be declared MIA) Something reasonable. Perhaps a few weeks to a month. Since this hasn't actually ever been a problem from what I can tell it doesn't matter much at this point. I had a quick look at the list of packages in question (http://hackage.haskell.org/trac/ghc/wiki/Repositories) and to my knowledge all these maintainers are around (and typically well-known contributors in the community.) >> I think the problem is one of misunderstanding how the >> process of managing dependencies ought to work (and how it works >> elsewhere.) "We must release a new version of so-and-so lib because we >> made such-and-such change" is wrong. Upstream changes (i.e. to GHC >> deps) ought to happen before downstream releases of dependent code >> (i.e. GHC.) > > This is actually the main reason that the situation between GHC and the > libraries it uses is different to most other packages, both within > Haskell and without: > > It is true that ghc depends on (for example) containers; but containers > also depends on base, and base/ghc are so intertwined that they are > essentially the same package (at least, I don't think you're suggesting > that we should make separate base and ghc releases). > > That is what I mean by them being part of the same system. > > For example, I recently removed the 'catch' export from Prelude, and > this required corresponding changes in Cabal, Win32 and haskeline. It's > not possible to make the change in the base library without making the > corresponding changes, or the GHC build would break, and there's no > reason the maintainers of the other packages would make the change if I > didn't ask them to. > > A more mundane example is library dependencies. If we make a change in > filepath that requires bumping its major version, then we need libraries > such as Cabal to relax their dependency on filepath or, again, the GHC > build would break. Go ahead and make those changes to your local clones. That's reasonable. That in no way forces you to make releases of those packages. You have the time from you make the changes to the next GHC release to get the changes pushed upstream and released. > Yes. Some of the upstreams respond so fast that it makes my head spin, > while others often either don't respond or continually promise to get to > things soon. (again, these are good, well-meaning people, who do a lot > for the community). Have you tried pointing out that this is a problem for GHC and perhaps suggest that they let you make a release on their behalf? >> I believe this is the first time I've seen an email of this kind >> from GHC HQ. > > Generally these mails are all directly to maintainers. They're generally > longer than this, but in essence it normally goes something like > mail 1: "Could you take a look at this patch please?" > mail 2: "Did you have a minute to look at that patch?" > mail 3: "I think the patch is good. Would it help if I pushed it for you?" > mail 4: "This is blocking other things, so I'd like to push it. > Please let me know within a week if you object" > (there may be multiple mail 2s, and mail 3 sometimes gets an > affirmative response). > > Once this has happened a few times, we tend to suggest switching to a > system where we just push by default, without the need for the mails and > the delay (in fact, more-or-less what Gershom suggested). If the maintainer is fine with such a solution this is of course fine. But what if they're not? Note that this has nothing to do with releasing from HEAD, which GHC has typically done in the past. Such patches typically only require a patch release. P.S. The Haskell Platform is most likely moving to the complete opposite approach of what GHC uses; a call for version bumps will go out some time before the release and if maintainers don't ask to have their package version bumped, it will stay at the same version used for the last release. Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Fwd: ghc-7.6 branch
| > Has maintainer's not being responsive been a problem for GHC in the | > past? | | Yes. Some of the upstreams respond so fast that it makes my head spin, | while others often either don't respond or continually promise to get to | things soon. (again, these are good, well-meaning people, who do a lot | for the community). The obvious solution, as someone else pointed out, is for someone at GHC HQ (perhaps Ian) to be a co-maintainer of these critical dependencies -- except perhaps for libraries whose maintainers are the "make your head spin" responsive kind. This would of course need the maintainer to trust the GHC person to push patches and make releases in sync with GHC. But my guess is they'd be willing. After all, we're all on the same side here! Would that cut the Geordian knot? Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
Hi Johan, On Wed, Jun 27, 2012 at 03:06:39PM -0700, Johan Tibell wrote: > > On Wed, Jun 27, 2012 at 12:53 PM, Ian Lynagh wrote: > > If a GHC release needs an unreleased change in one of the libraries, and > > the maintainer (for whatever reason) is not responding to e-mails, > > should the GHC release be held up indefinitely? You didn't give a clear answer to my question. Am I right in thinking that your answer would be "Yes, the GHC release should be delayed indefinitely"? (or at least, for long enough for the maintainer to be declared MIA) > Again, note that GHC is no different from any other package here. > > I think the problem is one of misunderstanding how the > process of managing dependencies ought to work (and how it works > elsewhere.) "We must release a new version of so-and-so lib because we > made such-and-such change" is wrong. Upstream changes (i.e. to GHC > deps) ought to happen before downstream releases of dependent code > (i.e. GHC.) This is actually the main reason that the situation between GHC and the libraries it uses is different to most other packages, both within Haskell and without: It is true that ghc depends on (for example) containers; but containers also depends on base, and base/ghc are so intertwined that they are essentially the same package (at least, I don't think you're suggesting that we should make separate base and ghc releases). That is what I mean by them being part of the same system. For example, I recently removed the 'catch' export from Prelude, and this required corresponding changes in Cabal, Win32 and haskeline. It's not possible to make the change in the base library without making the corresponding changes, or the GHC build would break, and there's no reason the maintainers of the other packages would make the change if I didn't ask them to. A more mundane example is library dependencies. If we make a change in filepath that requires bumping its major version, then we need libraries such as Cabal to relax their dependency on filepath or, again, the GHC build would break. > Has maintainer's not being responsive been a problem for GHC in the > past? Yes. Some of the upstreams respond so fast that it makes my head spin, while others often either don't respond or continually promise to get to things soon. (again, these are good, well-meaning people, who do a lot for the community). > I believe this is the first time I've seen an email of this kind > from GHC HQ. Generally these mails are all directly to maintainers. They're generally longer than this, but in essence it normally goes something like mail 1: "Could you take a look at this patch please?" mail 2: "Did you have a minute to look at that patch?" mail 3: "I think the patch is good. Would it help if I pushed it for you?" mail 4: "This is blocking other things, so I'd like to push it. Please let me know within a week if you object" (there may be multiple mail 2s, and mail 3 sometimes gets an affirmative response). Once this has happened a few times, we tend to suggest switching to a system where we just push by default, without the need for the mails and the delay (in fact, more-or-less what Gershom suggested). Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On 6/27/12 6:06 PM, Johan Tibell wrote: This is not a theoretical issue. We have had all of the following problems happen in the past due to the current process: * patches never making it upstream * releases of libraries without knowledge of the maintainer (who finds out by finding a new version of his/her package on Hackage.) * packages being released by GHC never ending up on Hackage, causing build breakages for people who use older GHCs and can't install the packages as they aren't available on Hackage. At the almost certain risk of stepping into a discussion I don't fully understand, let me step into a discussion I almost certainly don't fully understand :-) It seems to me that all these issues could be solved by having a member of the GHC team an assistant co-maintainer on packages that GHC depends on, and acting as such in a responsible manner, and in addition, having all packages bundled with GHC releases drawn from hackage releases. This is to say, that ghc-originated patches necessarily get committed to the upstream repo, because they must be there to be released on hackage, that ghc-originated patches necessarily get released to hackage because they must be there for GHC releases to draw on them, and maintainers necessarily know what gets released to hackage because they communicate well with co-maintainers. This is different than community ownership -- packages are still owned and maintained by individuals. However, by having a ghc assistant co-maintainer, there's a specified conduit for collaboration. This is also different from the current situation, because a co-maintainer may only work on issues for GHC release compatibility, but they are acting as someone with direct responsibility for the package and as part of the team that "owns" the package. Problems of collaboration aren't magiced away by this sort of change of titles, of course, but when there are problems of communication and collaboration, they can now be understood as and treated as problems between primary and secondary package maintainers. I hope this makes some semblance of sense. Cheers, Gershom ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
Hi, On Wed, Jun 27, 2012 at 12:53 PM, Ian Lynagh wrote: > If a GHC release needs an unreleased change in one of the libraries, and > the maintainer (for whatever reason) is not responding to e-mails, > should the GHC release be held up indefinitely? Again, note that GHC is no different from any other package here. If the maintainer is not responsive a person depending on his/her package can: 1) Try to convince him/her to become responsive. 2) Try to change the maintainer (this has happened for a few Hackage libraries, with the previous maintainer's approval usually.) 3) Drop the dependency. 4) Fork the package. This is how open source works outside GHC (including in different community's.) I don't see a need for doing anything differently here. Has maintainer's not being responsive been a problem for GHC in the past? I believe this is the first time I've seen an email of this kind from GHC HQ. > If the answer is "no", then there are going to be times when we need to > ship GHC with a version of a library that is not yet released. With the > best will in the world, there are always going to be people who are > swamped by real life, people on vacation, or even people who unbeknownst > to us have died. If people don't have time I'm sure they won't mind handing over maintenance to GHC HQ. I don't think it's OK to say "if you're on vacation and don't reply in X days we're making a release of your package." Imagine if someone did that on Hackage: "I really needed change X to your package to make my package work but since you didn't reply to my email I made the change and released a new version of your package." > But all that is really tangential to the main issue: even if the answer > to the above question is "no", that does not mean that we need to > routinely release libraries maintained by active upstreams. If upstream > is responsive, then we can discuss with them what code to use and what > releases need to be made. The original e-mail was intended to be the > first in that discussion. Perhaps we phrased it badly, or perhaps you > have bad memories of previous mistakes or of previous systems of > releasing, but all we were trying to do is to find out what code we > should set up the new stable branch to use. Phrasing aside, I think the problem is one of misunderstanding how the process of managing dependencies ought to work (and how it works elsewhere.) "We must release a new version of so-and-so lib because we made such-and-such change" is wrong. Upstream changes (i.e. to GHC deps) ought to happen before downstream releases of dependent code (i.e. GHC.) The current system is only possible due to GHC shipping libraries together with the compiler, several of it only uses internally to boot! This is not a theoretical issue. We have had all of the following problems happen in the past due to the current process: * patches never making it upstream * releases of libraries without knowledge of the maintainer (who finds out by finding a new version of his/her package on Hackage.) * packages being released by GHC never ending up on Hackage, causing build breakages for people who use older GHCs and can't install the packages as they aren't available on Hackage. Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Wed, Jun 27, 2012 at 11:42:24AM -0700, Johan Tibell wrote: > On Wed, Jun 27, 2012 at 8:12 AM, Ian Lynagh wrote: > > On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote: > >> > >> I just want to see things changed. :) > > > > We're happy to try to improve things, but I'm not sure what change you > > want exactly. > > I want GHC to stop releasing other people's code, unless they've > explicitly asked GHC to do so. Other than that, you can do what you > want. Here's an honest question: If a GHC release needs an unreleased change in one of the libraries, and the maintainer (for whatever reason) is not responding to e-mails, should the GHC release be held up indefinitely? If the answer is "yes", then perhaps we should go back to community maintainership for all the libraries that GHC ships with. As well as being entities in their own right, those libraries are also part of larger systems (ghc, and perhaps also other Haskell implementations). If the answer is "no", then there are going to be times when we need to ship GHC with a version of a library that is not yet released. With the best will in the world, there are always going to be people who are swamped by real life, people on vacation, or even people who unbeknownst to us have died. But all that is really tangential to the main issue: even if the answer to the above question is "no", that does not mean that we need to routinely release libraries maintained by active upstreams. If upstream is responsive, then we can discuss with them what code to use and what releases need to be made. The original e-mail was intended to be the first in that discussion. Perhaps we phrased it badly, or perhaps you have bad memories of previous mistakes or of previous systems of releasing, but all we were trying to do is to find out what code we should set up the new stable branch to use. We're happy to discuss concrete changes to the way things work. But (a) I don't think any change is necessary if your goal is for us to not make containers releases, and (b) GHC is a complex beast, and it already take lots of work over a period of weeks (or sometimes months) to get a release out. I'm keen to keep the process as lightweight as possible. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Wed, Jun 27, 2012 at 8:12 AM, Ian Lynagh wrote: > On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote: >> >> I just want to see things changed. :) > > We're happy to try to improve things, but I'm not sure what change you > want exactly. I want GHC to stop releasing other people's code, unless they've explicitly asked GHC to do so. Other than that, you can do what you want. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Wed, Jun 27, 2012 at 3:20 AM, Paolo Capriotti wrote: > On Wed, Jun 27, 2012 at 12:30 AM, Johan Tibell wrote: >>> * Some libraries will need to have version bumps, which means that other >>> libraries will need to loosen their dependencies, which means another >>> release will be needed anyway >> >> GHC is no different that any other library here though. Library A is >> released and thus library B needs to be updated and released. The >> argument here is that the author of library A needs to make a release >> of the author of library B's package. > > The difference is that here the dependency is mutual, in a way. GHC needs > library B to build, and library B needs to adapt to changes to GHC. Making a local change is not the same as making a release though. I make local changes to other people's code all the time, to unblock myself until the patches have made their way upstream. I'm not against that, I'm against releasing other people's code except they explicitly ask you to. > That's not what I wrote, however. We are just creating branches at this stage. > If we do make changes and need a version bump, we will coordinate again with > the maintainer to ensure everything goes upstream normally. I know that's not what you wrote (hence the "has not happened yet" part.) But consider the scenario for a second. If GHC adds a new package as a dependency now all of a sudden the maintainer of that package is in a situation where he/she explicitly needs to state a preference to have someone else not make release versions of his/her package! > It still makes sense for us to maintain a 'ghc-x.x' branch in each of the > mirrored resositories. It doesn't have to mirror an upstream branch > necessarily, but that's the easiest way to ensure all commits end up upstream. Except this is not what happens always. Commits have not always made it upstream in the past and sometimes packages released by GHC aren't uploaded to Hackage. > The current workflow is: > > - library needs a change > - make the change locally, validate, test > - send the patch upstream This doesn't always happen. > - wait for the patch to be integrated > - pull changes into the corresponding GHC mirror Just amend > - Before release, ask maintainer to make new release the package. and we'll be in the same situation of all other packages on Hackage. > Using tags as you propose should also be possible, but it would require a > release on the part of the library maintainer every time a change is made, and > then we need to update our mirror and point it to the new tag. I don't really object to you having unreleased changes for shorter periods of time. Just you releasing them. > The advantage would be that no revision of GHC ever depends an unreleased > revision of a library, but is this really worth the extra trouble? Those > intermediate releases are not particularly meaningful, and they may even end > up > being unusable with every released version of GHC. I don't think this is necessary. The problem is releasing other people's code, not having temporary deviating branches. P.S. All these problems will disappear once we have proper ACLs on Hackage, as at that point no one but the maintainer and people he/she gives permission to can upload his/her package to Hackage. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote: > > I just want to see things changed. :) We're happy to try to improve things, but I'm not sure what change you want exactly. We could change the default for GHC stable branches to: * Use the tag for the latest release, unless that isn't suitable, in which case use HEAD but note that it isn't necessarily easy to tell in advance if the latest release is suitable. For example, if library A decides to use a new major version number, then library B may need to relax its dependency on library A. Of course, we will make any decisions in discussion with the maintainers. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
(Moving lots of people to BCC. If you want to follow this discussion it will continue on the glasgow-haskell-users@haskell.org list.) On Tue, Jun 26, 2012 at 3:40 PM, Ian Lynagh wrote: > Please attribute any blame to me, not Paolo; he's only doing what I > asked him to :-) No blame to attribute. I just want to see things changed. :) > When following option 1, I think starting from the HEAD rather than the > last release makes sense in general: Maybe. It doesn't matter as I want (2). > * Some of the recent changes may have been to make the library work with > GHC HEAD, and will therefore be necessary to work with the 7.6 branch This works already today. I periodically get patches to e.g. network and containers to make it work with the current GHC HEAD. That seems to work fine. > * Some libraries will need to have version bumps, which means that other > libraries will need to loosen their dependencies, which means another > release will be needed anyway GHC is no different that any other library here though. Library A is released and thus library B needs to be updated and released. The argument here is that the author of library A needs to make a release of the author of library B's package. > so I think it's a reasonable default. But if it doesn't apply for a > particular library, then no problem, just let us know - that's why we're > e-mailing you :-) So at least don't make releases of containers (I think that's the only library I maintain that's released by GHC nowadays.) Can I please have this preference sticky in case I'm on vacation next time one of these emails go out? Aside: I don't believe this has happened yet but imagine the odd feeling of a library author, whose library was recently added as a GHC dependency, getting an email saying that if he/she doesn't reply GHC will make a release of his/her library! > That sounds like it would also work. I assume there's some way to pull > to a tag. git fetch (and thus pull) retrieves all branches and tags by default (and thus all commits pointed to by them.) Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
