Re: parallel garbage collection performance
On 26/06/2012 00:42, Ryan Newton wrote: However, the parallel GC will be a problem if one or more of your cores is being used by other process(es) on the machine. In that case, the GC synchronisation will stall and performance will go down the drain. You can often see this on a ThreadScope profile as a big delay during GC while the other cores wait for the delayed core. Make sure your machine is quiet and/or use one fewer cores than the total available. It's not usually a good idea to use hyperthreaded cores either. Does it ever help to set the number of GC threads greater than numCapabilities to over-partition the GC work? The idea would be to enable some load balancing in the face of perturbation from external load on the machine... It looks like GHC 6.10 had a -g flag for this that later went away? The GC threads map one-to-one onto mutator threads now (since 6.12). This change was crucial for performance, before that we hardly ever got any speedup from parallel GC because there was no guarantee of locality. I don't think it would help to have more threads. The load-balancing is already done with work-stealing, it isn't statically partitioned. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Compiling ghcjs
Hi, I hope this is the correct list to ask this question. I am trying to compile the ghcjs compiler. I am on ubuntu 12.04 and have ghc-7.4.1 installed (via apt-get). I am following the instruction I found here: https://github.com/ghcjs/ghcjs The first trouble comes with git pull ghcjs. I get: remote: Counting objects: 42, done. remote: Compressing objects: 100% (9/9), done. remote: Total 26 (delta 22), reused 21 (delta 17) Unpacking objects: 100% (26/26), done. From github.com:ghcjs/packages-Cabal * [new branch] encoding - ghcjs/encoding * [new branch] ghc-7.2- ghcjs/ghc-7.2 * [new branch] ghc-7.4- ghcjs/ghc-7.4 * [new branch] master - ghcjs/master You asked to pull from the remote 'ghcjs', but did not specify a branch. Because this is not the default configured remote for your current branch, you must specify a branch on the command line. So I am doing: git pull ghcjs ghc-7.4 Then git branch ghc-7.4 ghcjs/ghc-7.4 gives me: fatal: A branch named 'ghc-7.4' already exists. And finaly perl boot fails with: Error: libraries/extensible-exceptions/LICENSE doesn't exist. Maybe you haven't done './sync-all get'? at boot line 74, PACKAGES line 57. What can I do? Thanks! Nathan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote: I just want to see things changed. :) We're happy to try to improve things, but I'm not sure what change you want exactly. We could change the default for GHC stable branches to: * Use the tag for the latest release, unless that isn't suitable, in which case use HEAD but note that it isn't necessarily easy to tell in advance if the latest release is suitable. For example, if library A decides to use a new major version number, then library B may need to relax its dependency on library A. Of course, we will make any decisions in discussion with the maintainers. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Wed, Jun 27, 2012 at 3:20 AM, Paolo Capriotti p.caprio...@gmail.com wrote: On Wed, Jun 27, 2012 at 12:30 AM, Johan Tibell johan.tib...@gmail.com wrote: * Some libraries will need to have version bumps, which means that other libraries will need to loosen their dependencies, which means another release will be needed anyway GHC is no different that any other library here though. Library A is released and thus library B needs to be updated and released. The argument here is that the author of library A needs to make a release of the author of library B's package. The difference is that here the dependency is mutual, in a way. GHC needs library B to build, and library B needs to adapt to changes to GHC. Making a local change is not the same as making a release though. I make local changes to other people's code all the time, to unblock myself until the patches have made their way upstream. I'm not against that, I'm against releasing other people's code except they explicitly ask you to. That's not what I wrote, however. We are just creating branches at this stage. If we do make changes and need a version bump, we will coordinate again with the maintainer to ensure everything goes upstream normally. I know that's not what you wrote (hence the has not happened yet part.) But consider the scenario for a second. If GHC adds a new package as a dependency now all of a sudden the maintainer of that package is in a situation where he/she explicitly needs to state a preference to have someone else not make release versions of his/her package! It still makes sense for us to maintain a 'ghc-x.x' branch in each of the mirrored resositories. It doesn't have to mirror an upstream branch necessarily, but that's the easiest way to ensure all commits end up upstream. Except this is not what happens always. Commits have not always made it upstream in the past and sometimes packages released by GHC aren't uploaded to Hackage. The current workflow is: - library needs a change - make the change locally, validate, test - send the patch upstream This doesn't always happen. - wait for the patch to be integrated - pull changes into the corresponding GHC mirror Just amend - Before release, ask maintainer to make new release the package. and we'll be in the same situation of all other packages on Hackage. Using tags as you propose should also be possible, but it would require a release on the part of the library maintainer every time a change is made, and then we need to update our mirror and point it to the new tag. I don't really object to you having unreleased changes for shorter periods of time. Just you releasing them. The advantage would be that no revision of GHC ever depends an unreleased revision of a library, but is this really worth the extra trouble? Those intermediate releases are not particularly meaningful, and they may even end up being unusable with every released version of GHC. I don't think this is necessary. The problem is releasing other people's code, not having temporary deviating branches. P.S. All these problems will disappear once we have proper ACLs on Hackage, as at that point no one but the maintainer and people he/she gives permission to can upload his/her package to Hackage. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Wed, Jun 27, 2012 at 8:12 AM, Ian Lynagh i...@well-typed.com wrote: On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote: I just want to see things changed. :) We're happy to try to improve things, but I'm not sure what change you want exactly. I want GHC to stop releasing other people's code, unless they've explicitly asked GHC to do so. Other than that, you can do what you want. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On Wed, Jun 27, 2012 at 11:42:24AM -0700, Johan Tibell wrote: On Wed, Jun 27, 2012 at 8:12 AM, Ian Lynagh i...@well-typed.com wrote: On Tue, Jun 26, 2012 at 04:30:02PM -0700, Johan Tibell wrote: I just want to see things changed. :) We're happy to try to improve things, but I'm not sure what change you want exactly. I want GHC to stop releasing other people's code, unless they've explicitly asked GHC to do so. Other than that, you can do what you want. Here's an honest question: If a GHC release needs an unreleased change in one of the libraries, and the maintainer (for whatever reason) is not responding to e-mails, should the GHC release be held up indefinitely? If the answer is yes, then perhaps we should go back to community maintainership for all the libraries that GHC ships with. As well as being entities in their own right, those libraries are also part of larger systems (ghc, and perhaps also other Haskell implementations). If the answer is no, then there are going to be times when we need to ship GHC with a version of a library that is not yet released. With the best will in the world, there are always going to be people who are swamped by real life, people on vacation, or even people who unbeknownst to us have died. But all that is really tangential to the main issue: even if the answer to the above question is no, that does not mean that we need to routinely release libraries maintained by active upstreams. If upstream is responsive, then we can discuss with them what code to use and what releases need to be made. The original e-mail was intended to be the first in that discussion. Perhaps we phrased it badly, or perhaps you have bad memories of previous mistakes or of previous systems of releasing, but all we were trying to do is to find out what code we should set up the new stable branch to use. We're happy to discuss concrete changes to the way things work. But (a) I don't think any change is necessary if your goal is for us to not make containers releases, and (b) GHC is a complex beast, and it already take lots of work over a period of weeks (or sometimes months) to get a release out. I'm keen to keep the process as lightweight as possible. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
Hi, On Wed, Jun 27, 2012 at 12:53 PM, Ian Lynagh ig...@earth.li wrote: If a GHC release needs an unreleased change in one of the libraries, and the maintainer (for whatever reason) is not responding to e-mails, should the GHC release be held up indefinitely? Again, note that GHC is no different from any other package here. If the maintainer is not responsive a person depending on his/her package can: 1) Try to convince him/her to become responsive. 2) Try to change the maintainer (this has happened for a few Hackage libraries, with the previous maintainer's approval usually.) 3) Drop the dependency. 4) Fork the package. This is how open source works outside GHC (including in different community's.) I don't see a need for doing anything differently here. Has maintainer's not being responsive been a problem for GHC in the past? I believe this is the first time I've seen an email of this kind from GHC HQ. If the answer is no, then there are going to be times when we need to ship GHC with a version of a library that is not yet released. With the best will in the world, there are always going to be people who are swamped by real life, people on vacation, or even people who unbeknownst to us have died. If people don't have time I'm sure they won't mind handing over maintenance to GHC HQ. I don't think it's OK to say if you're on vacation and don't reply in X days we're making a release of your package. Imagine if someone did that on Hackage: I really needed change X to your package to make my package work but since you didn't reply to my email I made the change and released a new version of your package. But all that is really tangential to the main issue: even if the answer to the above question is no, that does not mean that we need to routinely release libraries maintained by active upstreams. If upstream is responsive, then we can discuss with them what code to use and what releases need to be made. The original e-mail was intended to be the first in that discussion. Perhaps we phrased it badly, or perhaps you have bad memories of previous mistakes or of previous systems of releasing, but all we were trying to do is to find out what code we should set up the new stable branch to use. Phrasing aside, I think the problem is one of misunderstanding how the process of managing dependencies ought to work (and how it works elsewhere.) We must release a new version of so-and-so lib because we made such-and-such change is wrong. Upstream changes (i.e. to GHC deps) ought to happen before downstream releases of dependent code (i.e. GHC.) The current system is only possible due to GHC shipping libraries together with the compiler, several of it only uses internally to boot! This is not a theoretical issue. We have had all of the following problems happen in the past due to the current process: * patches never making it upstream * releases of libraries without knowledge of the maintainer (who finds out by finding a new version of his/her package on Hackage.) * packages being released by GHC never ending up on Hackage, causing build breakages for people who use older GHCs and can't install the packages as they aren't available on Hackage. Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Fwd: ghc-7.6 branch
On 6/27/12 6:06 PM, Johan Tibell wrote: This is not a theoretical issue. We have had all of the following problems happen in the past due to the current process: * patches never making it upstream * releases of libraries without knowledge of the maintainer (who finds out by finding a new version of his/her package on Hackage.) * packages being released by GHC never ending up on Hackage, causing build breakages for people who use older GHCs and can't install the packages as they aren't available on Hackage. At the almost certain risk of stepping into a discussion I don't fully understand, let me step into a discussion I almost certainly don't fully understand :-) It seems to me that all these issues could be solved by having a member of the GHC team an assistant co-maintainer on packages that GHC depends on, and acting as such in a responsible manner, and in addition, having all packages bundled with GHC releases drawn from hackage releases. This is to say, that ghc-originated patches necessarily get committed to the upstream repo, because they must be there to be released on hackage, that ghc-originated patches necessarily get released to hackage because they must be there for GHC releases to draw on them, and maintainers necessarily know what gets released to hackage because they communicate well with co-maintainers. This is different than community ownership -- packages are still owned and maintained by individuals. However, by having a ghc assistant co-maintainer, there's a specified conduit for collaboration. This is also different from the current situation, because a co-maintainer may only work on issues for GHC release compatibility, but they are acting as someone with direct responsibility for the package and as part of the team that owns the package. Problems of collaboration aren't magiced away by this sort of change of titles, of course, but when there are problems of communication and collaboration, they can now be understood as and treated as problems between primary and secondary package maintainers. I hope this makes some semblance of sense. Cheers, Gershom ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
