Re: Windows breakage -- again
2014-07-21 21:31 GMT+02:00 Johan Tibell : > Great. Thanks all for your help! I am afraid we are not done with this yet. Yesterday I have also committed the fix for the FreeBSD platform, but today I noticed that the corresponding test case ("AtomicPrimops") is failing due to SIGILL, that is, illegal instruction. And it has been happening for all the 32-bit platforms, including Linux [1], SmartOS [2], and Solaris [3]. I do not know yet why it goes wrong. [1] http://haskell.inf.elte.hu/builders/validator1-linux-x86-head/34/10.html [2] http://haskell.inf.elte.hu/builders/smartos-x86-head/73/21.html [3] http://haskell.inf.elte.hu/builders/solaris-x86-head/116/21.html ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: GHC contribution guidelines and infrastructure talk on 6th September at HIW?
On a related front... I don't have a talk to give (hence I didn't submit a proposal)... But I'd love it if some of us could have a group discussion about coordinating releases, and our approach to putting out "Haskell": In short, we see it as several related peices (GHC, Cabal, Haddock, core libs, platform, etc...) but my guess is that most developers considering using Haskell see it as one thing: "Can I haz the Haskellz on my machine? kthxbai?" Therefore, I think we could put some thought into how we manage these pieces into a cohesive whole whose release more or less "just works". Not sure if this should be a "session", a "workshop", a long hallway disucssion, a night of good food and beer, or what. I'm happy to put some effort into organizing, and setting the context for the discussion. - Mark ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: GHC contribution guidelines and infrastructure talk on 6th September at HIW?
On 18 July 2014 09:01, Joachim Breitner wrote: > Am Freitag, den 18.07.2014, 07:25 + schrieb Simon Peyton Jones: >> | On Saturday 6th September is the Haskell Implementers Workshop. There >> | has been plenty of discussion over the last 12 months about making >> | contributions to GHC less formidable. Is this story going to be told at >> | HIW? A talk about revised contribution guidelines and helpful tool >> | support might engage those sat on, or peering over, the fence. >> >> I think that's a great idea. Maybe Simon M, or Joachim, or Austin, or >> Herbert? Of some coalition thereof > > I agree, and I’d be available for it, or for joining a coalition. I gentle nudge about the idea of a HIW talk on contributing to GHC development. I'm glad some people think that this is a good idea. However, given that the official deadline for talk proposals has already passed, at least an abstract would have to be submitted to the HIW committee very soon to be considered. The presentation content can of course be put together much closer to the time. I don't want to create work for anyone, of course. -- Rob ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Thread status constants
Hi Simon, I have been inspecting what gcc generates and found that in all the functions I checked which used the thread status in a switch, none of them generated jump tables, rather, long chains of cmp's. I predict this is because there are overlapping cases (or even a fall-through in one I saw, which looked accidental...). On Tue, Jul 22, 2014 at 5:32 AM, Simon Marlow wrote: > On 19/07/2014 06:00, Kyle Van Berendonck wrote: > >> I found these: >> https://github.com/ghc/ghc/blob/5f3c5384df59717ca8013c5df8d1f6 >> 5692867825/includes/rts/Constants.h#L194 >> >> They go only 0-14, so there's some long chains of branches and stuff in >> hot paths that could be cleaned up into single &-masked branches by >> changing these into a set of flags. >> > > GHC will generate a table jump for a large dense case, if that's what > you're worried about. > > Cheers, > Simon > ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Beta Performance dashboard
Hi, Am Montag, den 21.07.2014, 20:30 +0100 schrieb Simon Marlow: > This is great! Can it do alerts, e.g. send a mail to the list when a > metric moves by a certain amount? no (or not yet). Large deviations are listed in the summary on http://ghcspeed-nomeata.rhcloud.com/, and I try to keep an eye on them. But the whole thing needs to settle first anyways. For example, in order to get the "unexpected failures" to zero we need to find an agreement in #9315 (where input from others would be helpful as well) The last few reports may be a bit noisy as I am re-measuring them with changed build settings. Greetings, Joachim -- Joachim Breitner e-Mail: m...@joachim-breitner.de Homepage: http://www.joachim-breitner.de Jabber-ID: nome...@joachim-breitner.de signature.asc Description: This is a digitally signed message part ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Multi-instance packages status report
Hi, Am Montag, den 21.07.2014, 18:43 +0100 schrieb Edward Z.Yang: > In my copy of GHC and Cabal [1,2], you can now install multiple copies of a > package with differing dependencies to the package database, i.e. q-1.0 > compiled against p-1.0, and against p-2.0. The packages in the database > are distinguished via a *package key*, which is an md5 hash of the > package id (e.g. q-1.0) and the sorted list of the package IDs of the > transitive closure of dependencies (e.g. p-1.0 or p-2.0). maybe a stupid question, but how does the package key relate to the hash that "ghc-pkg" shows for package? Thanks, Joachim -- Joachim “nomeata” Breitner m...@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nome...@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nome...@debian.org signature.asc Description: This is a digitally signed message part ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Thread status constants
On 19/07/2014 06:00, Kyle Van Berendonck wrote: I found these: https://github.com/ghc/ghc/blob/5f3c5384df59717ca8013c5df8d1f65692867825/includes/rts/Constants.h#L194 They go only 0-14, so there's some long chains of branches and stuff in hot paths that could be cleaned up into single &-masked branches by changing these into a set of flags. GHC will generate a table jump for a large dense case, if that's what you're worried about. Cheers, Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Windows breakage -- again
Great. Thanks all for your help! On Sat, Jul 19, 2014 at 2:04 AM, Austin Seipp wrote: > Thanks Niklas, this is now committed. > > On Fri, Jul 18, 2014 at 9:21 AM, Niklas Larsson wrote: >> I posted a working and tested patch last night. Please feel free to commit >> it, I haven't the rights to do it. >> >> Niklas >> >> Från: Simon Peyton Jones >> Skickat: 2014-07-18 15:55 >> Till: Niklas Larsson; Johan Tibell >> Kopia: ghc-devs@haskell.org >> Ämne: RE: Windows breakage -- again >> >> Thank you all for pursuing this. I gather that you know what is going on, >> so no further info needed from me. Yell if it is otherwise. >> >> >> >> Meanwhile, is the fix imminent, or should we revert Johan’s patch? >> >> >> >> Simon >> >> >> >> From: Niklas Larsson [mailto:metanik...@gmail.com] >> Sent: 16 July 2014 19:58 >> To: Johan Tibell; Simon Peyton Jones >> Cc: ghc-devs@haskell.org >> Subject: Re: Windows breakage -- again >> >> >> >> I get the same failure when I try to build HEAD. Turns out the error occurs >> on the 32-bit Windows build, and my successful build was a 64-bit build. My >> 64-bit build still succeeds. >> >> Also, gcc is 4.5.2 on 32-bit, not 4.6.3 as on 64-bit. >> >> Niklas >> >> >> >> >> >> 2014-07-16 14:48 GMT+02:00 Niklas Larsson : >> >> I have built ghc on windows after that was added with no issue. >> >> I can take a look this evening and see how HEAD works for me. >> >> The standard gcc in the tarballs is 4.6.3, which is getting long in the >> tooth, there is an issue on trac to upgrade it. >> >> -- Niklas >> >> >> >> Från: Johan Tibell >> Skickat: 2014-07-16 09:57 >> Till: Simon Peyton Jones >> Kopia: ghc-devs@haskell.org >> Ämne: Re: Windows breakage -- again >> >> You can rollback the commit (git revert >> 4ee4ab01c1d97845aecb7707ad2f9a80933e7a49) and push that to the repo if you >> wish. I will try to re-add the primop again after I figure out what's wrong. >> >> >> >> On Wed, Jul 16, 2014 at 9:37 AM, Johan Tibell >> wrote: >> >> I added some primops about a month ago >> (4ee4ab01c1d97845aecb7707ad2f9a80933e7a49) that call __sync_fetch_and_add, a >> gcc/llvm builtin. I'm a bit surprised to see this error. The GCC manual [1] >> says: >> >> >> >>> " Not all operations are supported by all target processors. If a >>> particular operation cannot be implemented on the target processor, a >>> warning will be generated and a call an external function will be generated. >>> The external function will carry the same name as the builtin, with an >>> additional suffix `_n' where n is the size of the data type." >> >> >> >> I'm a bit surprised by this error for two reasons: >> >> >> >> * A call to that symbol should only be generated if the CPU doesn't support >> the atomic instructions. What CPU model does Windows report that you have? >> >> >> >> * gcc should define such a symbol. For me the following test program >> compiles: >> >> >> >> #include >> >> >> >> uint8_t test(uint8_t* ptr, uint8_t val) { >> >> return __sync_fetch_and_add_1(ptr, val); >> >> } >> >> >> >> int main(void) { >> >> uint8_t n; >> >> return test(&n, 1); >> >> } >> >> >> >> Does that compile for you? Which version of GCC do we end up using on >> Windows? >> >> >> >> The reported symbol (___sync_fetch_and_add_1) has three leading underscores, >> that looks weird. Can you compile just libraries/ghc-prim/cbits/atomic.c and >> see if it's indeed GCC that generates a reference to that symbol? >> >> >> >> 1. http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Atomic-Builtins.html >> >> >> >> On Wed, Jul 16, 2014 at 12:29 AM, Simon Peyton Jones >> wrote: >> >> Aargh! The Windows build has broken – again. I can’t build GHC on my >> laptop any more. >> >> >> [Hela det ursprungliga meddelandet tas inte med.] >> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://www.haskell.org/mailman/listinfo/ghc-devs >> > > > > -- > Regards, > > Austin Seipp, Haskell Consultant > Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Beta Performance dashboard
This is great! Can it do alerts, e.g. send a mail to the list when a metric moves by a certain amount? Cheers, Simon On 16/07/2014 09:02, Joachim Breitner wrote: Hi, I guess it’s time to talk about this, especially as Richard just brought it up again... I felt that we were seriously lacking in our grip on performance issues. We don’t even know whether 6.8.3 was better or worse than 6.8.3 or 7.6.4 in terms of nofib, not to speak of the effect of each single commit. I want to change that, so I set up a benchmark monitoring dashboard. You can currently reach it at: http://ghcspeed-nomeata.rhcloud.com/ What does it do? It monitors the repository (master branch only) and builds each commit, complete with the test suite and nofib. The log is saved and analyzed, and some numbers are extracted: * The build time * The test suite summary numbers * Runtime (if >1s), allocations and binary sizes of the nofib benchmarks These are uploaded to the website above, which is powered by codespeed, a general performance dashboard, implemented in Python using Django. Under _Changes_, it provides a report for each commit (changes wrt. to the previous version, and wrt. to 10 revisions earlier, the so-called “trend”). A summary of these reports is visible on the front-page. The _Timeline_ is a graph for each individual performance number. If there are bumps, you can hopefully find them there! You can also compare to 7.8.3, which is available as a “baseline”. _Comparison_ will be more useful if we have more tagged revision, or if were benchmarking various options (e.g. -fllvm): Here you can do bar-chart comparisons. Why codespeed? ~~ For a long time I searched for a suitable software product, and one criterion is that it should be open source, rather simple to set up and mostly decoupled from other tools, i.e. something that I throw numbers at and which then displays them nicely. While I don’t think codespeed is the best performance dashboard out there (I find http://goperfd.appspot.com/perf a bit better; I wonder how well codespeed scales to even larger numbers of benchmarks and I wish it were more git-aware), it was the easiest to get started with. And thanks to the loose coupling of (1) running the tests to acquire a log, (2) parsing the log to get numbers and (3) putting them on a server, we can hopefully replace it when we come along something better. I was hoping for the Phabricator guys to have something in their tool suite, but doesn’t look like it. How does it work (currently)? ~ My office PC is underused (I work on my laptop), so its currently dedicated to it. I have a simple shell script that monitors the repo for new versions. It builds the newest revision and works itself back to the commit where everything was turned into submodules: https://github.com/nomeata/codespeed/blob/ghc/tools/ghc/watch.sh It calls a script that does the actual building: https://github.com/nomeata/codespeed/blob/ghc/tools/ghc/run-speed.sh This produces a log file which should contain all the required numbers somewhere. A second script extracts these numbers (with help of nofib-analyze) and converts them into codespeed compatible JSON files: https://github.com/nomeata/codespeed/blob/ghc/tools/ghc/log2json.pl Finally, a simple invocation to curl uploads them to codespeed: https://github.com/nomeata/codespeed/blob/ghc/tools/ghc/upload.sh So if you want additional benchmarks to be tracked, make sure they are present in the logs and adjust log2json.pl. codespeed will automatically pick up new benchmarks in these logs. Reimplementations in Haskell are also welcome :-) The testsuite is run with VERBOSE=4, so the performance numbers are also shown for failing test cases. So once a test case goes over the limit, you can grep through previous logs try to find the real culprit. I uploaded the logs (so far) to https://github.com/nomeata/ghc-speed-logs (but this is not automated yet, ping me if you need an update on this). What next? ~~ Clearly, the current setup is only good enough to evaluate the system. Eventually, I might want to use my office PC again, and the free hosting on openshift is not very powerful. So if we want to keep this setup and make it “official”, we need find a permanent solution.¹ This involves: * A dedicated machine to run the benchmarks. This probably shouldn’t be a VM, if we want to keep the noise in the runtime down. * A machine to run the codespeed server. Can be a VM, or even run on any of the system that we have right now. Just needs a database (postgresql preferably) and a webserver supporting WSGI (i.e. any of them). * Maybe a better place to store the logs for public consumption. Also, there are way to improve the system: * As I said, I don’t think codespeed is the best. If we find something better, we can replace it. Since we have all the logs, we can easily fill the n
Multi-instance packages status report
Hello all, As you may have noticed, I've been knocking around GHC and Cabal the past few weeks. One of the tasks that has been on my list is essentially reimplementing Philipp Schuster's 2012 GSoC, with a few small but important architectural differences. Here is a status report of what is going on. In my copy of GHC and Cabal [1,2], you can now install multiple copies of a package with differing dependencies to the package database, i.e. q-1.0 compiled against p-1.0, and against p-2.0. The packages in the database are distinguished via a *package key*, which is an md5 hash of the package id (e.g. q-1.0) and the sorted list of the package IDs of the transitive closure of dependencies (e.g. p-1.0 or p-2.0). The package key is used to generate linker symbols for packages, so it's possible to link together both copies of q in the same program, as long as you rename the modules appropriately (of course, the redefined types are considered unequal). When registering a package, ghc-pkg now checks and removes duplicate package keys, as opposed to package IDs. I've also implemented many of the necessary niceties for making it pleasant to deal with duplicated package IDs: for example, GHC's output logic has been adjusted to only qualify a package ID with the package key when there are multiples of the same package ID exposed in the database. I've also adjusted GHC's build system to use package keys rather than package IDs to refer to packages when building. Like the original GSoC project, we still need to have decisions on some of the major design choices. I think the most pressing one is "simplistic dependency resolution" when you use, e.g. ghc -package foo-1.0 or ./Setup configure --with-constraint="foo==1.0". Right now, I pick dependencies in an unspecified manner, check if they are consistent, and bail out if they are not. As it turns out, both GHC and ./Setup configure will already compute the transitive closure of dependencies, so I suspect we might be able to do something clever here. Less pressing but eventually necessary is clueing in cabal-install. Comments would be especially appreciated. Cheers, Edward [1] https://phabricator.haskell.org/D80 [2] https://github.com/haskell/cabal/pull/2002 ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Building GHC under Wine?
On 18/07/2014 08:59, Joachim Breitner wrote: Hi, Am Freitag, den 18.07.2014, 08:38 +0100 schrieb Simon Marlow: I might be misremembering, but I believe someone (Ross Paterson?) used to do this a while ago. I can't think of any good reasons it *shouldn't* work. Then, next question: Is it likely to find windows building failures, are are the failures usually of the kind that would not occur in a not-quite-a-real-windows environment? As far as I know, yes it should look like a real Windows environment. However, now that I think about it, I suspect what I'm remembering is that someone used GHC under Wine to develop something else, rather than building GHC itself. Building GHC might be rather more difficult. Cheers, Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Status updates
Hello *, Here are some updates from this past week. - I sent an email earlier about 7.10.1 tickets - please vote on them! - Related to that - as I said last week, I finished remilestoning a lot of tickets, but more remain. Please excuse any mail spam, but do pay attention to your bugs if they change! - I cleaned up some tabs and whitespace in the compiler in low-activity areas. You may have seen the commits go by, but otherwise it shouldn't be a problem. I'll probably keep eliminating errors in low-contention areas of the codebase as I can. Ditto with 'arc lint'. - I've cleaned up a bunch of wiki pages relating to the git repositories, and submodule management, but more stuff remains. See below for my reply to Simon. - I've been lazy about the Applicative-Monad work, since some of it fell out of date. But now other work is pending on it, so it's time to finish it Real Soon Now. - I'm working out some of the kinks in the Phabricator automation for builds etc, but it's not quite done yet - sorry! Here are some things I plan to do this week, roughly in order of priority: - I'm going to keep cleaning up these Git pages to make them more sensible, since some of the information is still wrong. Please see the comments below this and let me know what's wrong! - I will finalize these Phabricator things, which I should be able to get done with soon. This is all mostly annoying HTTP endpoint and PHP/Python stuff and is a bit of a chore to test/setup completely. - I am going to spend time next on landing AMP and get it back into shape. Yes, really landing it; I need to send something upstream to Cabal still. - The patch queue on Trac needs to be drained and slipped my mind. - Given all that I'll also probably end up continuing to fix lint errors, and hopefully fix bugs based on what people want in 7.10. :) Below is a quick recap of what me and Simon discussed last week, mostly stuff from the above: > · Complete re-milestoning. Then email ghc-devs and ghc-users [send a > draft to Simon & Simon first] > >o Giving links to suitable query(s) > >o Saying that any ticket that isn’t > >a) milestoned 7.8.4 or 7.10.1, and > >b) high or highest priority, > >isn’t going to get attended to by us. > >o Inviting them to bid (on the ticket) for high/highest 7.10.1 status, >explaining why it is important to them. And that it’s more likely to be >done if they help. I sent this mail earlier - please bid on 7.10.1 tickets! > · Trac/Phab integration Doing this in a complete manner is turning out to be a bit of work and I haven't gotten done yet with having comments appear on tickets just yet. My goal is to have it so that when you say 'arc diff', you can put the words: Trac: #9832, #9123 into the review, and it will auto appear on Phabricator as a custom field, hyperlink it, and then a comment will appear on Trac. I have to write some PHP and Python on both ends to get this all working, though. Also the builder script is currently really crappy, so I'm fixing that a bit, so it will correctly deal with concurrent builds and multiple builders. > · Document how to update a submodule; e.g. haddock > > o We worked through an example > > o We need a canonical place on the wiki to say which branch GHC should be > tracking on the submodule; and how to check that it is > > o Need to talk to haddock and cabal folk about whether GHC should track > master or a branch in their repo. They may want a more stable branch to work > on. > > > · Wiki pages > > o Be sure to update > https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources, to include > simple cloning story > > o Consolidate “how to use Git” information into WorkingConventions/Git, > perhaps headed “Git workflows”. Merging WorkingConventions/Git with the > Git/Submodules page will make it intimidatingly long. > > § Think about how to split it up. > > § Tricks should be on a separate page. > > § Focus on tasks, not technology > > o wiki:Repositories page says where everything lives, but not how to use > them. > > § Needs updating to have the correct data. > > § Also integrate https://ghc.haskell.org/trac/ghc/wiki/Repositories/Upstream This is now done with a first draft. I think I simplified the pages a lot. See here: https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Git and: https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Git/Submodules I plan on obsoleting the old Repositories pages since they're mostly redundant and out of date now. There may be more info stuffed into other places, but I imagine it will stay under this hierarchy. There are still some links and prose that need to be rewritten that I obviously missed however. > · Merge AMP stuff. Should be done now. (Without join in Monad for > now, #9123.) https://phabricator.haskell.org/D13 Unfortunately this is still pending a few things, so I haven't gotten it done yet. :( > o
GHC 7.10.1 tickets
Hello *, We need to be sorting out tickets for 7.10.1. Last week, I spent some time remilestoning things to get them into the rough correct places (roughly), but we all need some help to get an idea of what we should fix for the release. Here's a query to look at: https://ghc.haskell.org/trac/ghc/query?status=!closed&milestone=7.10.1&order=priority I have already categorized these tickets roughly, but we'll probably still address some of the priorities on existing tickets of course. These are all of the 7.10.1 tickets currently, but there are some important things to note: 1) There are a lot of them! 2) Most of the development team is probably not going to pay attention to many of them unless they are high or highest priority in Trac. (That's just the result of having so many tickets for 7.10.1 at the moment.) So I'm asking if people out there with the time can help chip in and give some input on the ticket! - If there's a ticket that interests you, please comment on it! - Say why you think it should have its priority elevated for 7.10.1. - It will certainly be easier if you offer to help. Then we can move it upwards and we'll look at it. Finally, if you are *really* dedicated or a newcomer - please go through the tickets and see if any of them can be closed, reassigned, marked as duplicates, etc. There are certainly some dead tickets and probably easy tickets for newcomers, so just spend a few minutes looking if you're interested! (Otherwise they just end up getting punted off in batch modifications) Thanks! ~~~ BTW, for developers, you can look at your assigned tickets for 7.10.1 here, it's also always on the sidebar: https://ghc.haskell.org/trac/ghc/query?owner=%24USER&status=infoneeded&status=merge&status=new&status=patch&milestone=7.10.1&order=priority -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Update Cabal submodule to HEAD (1.21)
Hi, Am Sonntag, den 20.07.2014, 22:57 +0100 schrieb Edward Z.Yang: > Since this patch causes GHC HEAD to not bootstrap out of the box > from GHC 7.6, I've reverted it for now. We'll have to cross > this bridge sometime though. thanks. Cabal already has applied the patch to make the initial ghc-cabal binary build: https://github.com/haskell/cabal/commit/3ef560208721a050e91fd9e67a0066ce44b04ba2 Now all we need to do is to figure out how to tell the build system to build process before Cabal. I started in http://www.haskell.org/pipermail/ghc-devs/2014-July/005685.html but that didn’t satisfiy the build system completely. Probably simply, but I have been staring at the makefiles for too long already. Greetings, Joachim -- Joachim “nomeata” Breitner m...@joachim-breitner.de • http://www.joachim-breitner.de/ Jabber: nome...@joachim-breitner.de • GPG-Key: 0xF0FBF51F Debian Developer: nome...@debian.org signature.asc Description: This is a digitally signed message part ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs