Re: Validating with Haddock
Hi Mateusz, I've pushed your work and tweaked the testsuite performance numbers on 64bit. The 32bit ones are out of date, but I'll fix them shortly. I also fixed some of the documentation errors. Thanks for all your hard work. On Fri, Jan 10, 2014 at 4:06 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 10/01/14 10:01, Mateusz Kowalczyk wrote: Hi all, I have now merged in the new parser and new features onto a single branch. I'm having some issues validating with HEAD at the moment (#8661, unrelated problem) but while I get that sorted out, someone might want to try validating with Haddock changes on their own platform. The full branch is at [1]. I have squashed the changes to what I feel is the minimum number of commits until they completely stop making sense. It should apply cleanly on top of current Haddock master branch. The documentation is updated so you can read about what changed. Feel free to ask any questions. I will post again once I can confirm that the branch validates for me without any new test failures. Thanks for your patience. [1]: https://github.com/Fuuzetsu/haddock/tree/new-features This is just a simple follow up to say that the changes don't seem to break anything new on 32-bit Linux. I provide my validate logs before[1] and after[2] Haddock changes. Here's a word of warning: previously, when the mark-up wasn't 100% clear, we'd get a parse error and no documentation for the whole package. The new parser no longer does this and instead does its best to parse and present everything. This means that any Haddock parse failures should be reported as bugs. As you can see in [1], there were some parse failures in the past (look for ‘doc comment parse failed’) and they will now be rendered. This means the documentation might look bad in those places so it's probably worth while visiting those places and having a look. On an upside, at least we now have documentation for those packages. Validation was ran on commit 15a3de1288fe9d055f3dc92d554cb59b3528fa30 including #8661 fixes. Here's the relevant tail of the logs: Unexpected results from: TEST=lazy-bs-alloc T1969 T3064 T4801 T3294 T5498 haddock.Cabal haddock.compiler haddock.base OVERALL SUMMARY for test run started at Fri Jan 10 12:45:39 2014 GMT 0:17:23 spent to go through 3861 total tests, which gave rise to 15072 test cases, of which 11547 were skipped 28 had missing libraries 3432 expected passes 56 expected failures 0 caused framework failures 0 unexpected passes 9 unexpected failures Unexpected failures: deriving/should_fail T5498 [stderr mismatch] (normal) perf/compiler T1969 [stat too good] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T3294 [stat not good enough] (normal) perf/compiler T4801 [stat not good enough] (normal) perf/haddock haddock.Cabal [stat not good enough] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/haddock haddock.compiler [stat not good enough] (normal) perf/should_run lazy-bs-alloc [stat too good] (normal) gmake[2]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' gmake[1]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' == Start post-testsuite package check Timestamp 2014-01-10 12:45:36.897842164 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache Timestamp 2014-01-10 12:45:36 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d (older than cache) using cache: /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache == End post-testsuite package check --- Oops! Looks like you have some unexpected test results or framework failures. Please fix them before pushing/sending patches. --- The failures are the same in both logs. Thanks! [1]: http://fuuzetsu.co.uk/misc/segfix [2]: http://fuuzetsu.co.uk/misc/segfixhaddock -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
Hi all, I have now merged in the new parser and new features onto a single branch. I'm having some issues validating with HEAD at the moment (#8661, unrelated problem) but while I get that sorted out, someone might want to try validating with Haddock changes on their own platform. The full branch is at [1]. I have squashed the changes to what I feel is the minimum number of commits until they completely stop making sense. It should apply cleanly on top of current Haddock master branch. The documentation is updated so you can read about what changed. Feel free to ask any questions. I will post again once I can confirm that the branch validates for me without any new test failures. Thanks for your patience. [1]: https://github.com/Fuuzetsu/haddock/tree/new-features -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 10/01/14 10:01, Mateusz Kowalczyk wrote: Hi all, I have now merged in the new parser and new features onto a single branch. I'm having some issues validating with HEAD at the moment (#8661, unrelated problem) but while I get that sorted out, someone might want to try validating with Haddock changes on their own platform. The full branch is at [1]. I have squashed the changes to what I feel is the minimum number of commits until they completely stop making sense. It should apply cleanly on top of current Haddock master branch. The documentation is updated so you can read about what changed. Feel free to ask any questions. I will post again once I can confirm that the branch validates for me without any new test failures. Thanks for your patience. [1]: https://github.com/Fuuzetsu/haddock/tree/new-features This is just a simple follow up to say that the changes don't seem to break anything new on 32-bit Linux. I provide my validate logs before[1] and after[2] Haddock changes. Here's a word of warning: previously, when the mark-up wasn't 100% clear, we'd get a parse error and no documentation for the whole package. The new parser no longer does this and instead does its best to parse and present everything. This means that any Haddock parse failures should be reported as bugs. As you can see in [1], there were some parse failures in the past (look for ‘doc comment parse failed’) and they will now be rendered. This means the documentation might look bad in those places so it's probably worth while visiting those places and having a look. On an upside, at least we now have documentation for those packages. Validation was ran on commit 15a3de1288fe9d055f3dc92d554cb59b3528fa30 including #8661 fixes. Here's the relevant tail of the logs: Unexpected results from: TEST=lazy-bs-alloc T1969 T3064 T4801 T3294 T5498 haddock.Cabal haddock.compiler haddock.base OVERALL SUMMARY for test run started at Fri Jan 10 12:45:39 2014 GMT 0:17:23 spent to go through 3861 total tests, which gave rise to 15072 test cases, of which 11547 were skipped 28 had missing libraries 3432 expected passes 56 expected failures 0 caused framework failures 0 unexpected passes 9 unexpected failures Unexpected failures: deriving/should_fail T5498 [stderr mismatch] (normal) perf/compiler T1969 [stat too good] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T3294 [stat not good enough] (normal) perf/compiler T4801 [stat not good enough] (normal) perf/haddock haddock.Cabal [stat not good enough] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/haddock haddock.compiler [stat not good enough] (normal) perf/should_run lazy-bs-alloc [stat too good] (normal) gmake[2]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' gmake[1]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' == Start post-testsuite package check Timestamp 2014-01-10 12:45:36.897842164 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache Timestamp 2014-01-10 12:45:36 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d (older than cache) using cache: /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache == End post-testsuite package check --- Oops! Looks like you have some unexpected test results or framework failures. Please fix them before pushing/sending patches. --- The failures are the same in both logs. Thanks! [1]: http://fuuzetsu.co.uk/misc/segfix [2]: http://fuuzetsu.co.uk/misc/segfixhaddock -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
RE: Validating with Haddock
I get a different bunch of post-build package check complaints. Does anyone else have a clue what is going on? I do not. Mine are reproduced below. They appear to be non-fatal warnings. I bet it's because I have HADDOCK_DOCS=NO, but if so that should surely suppress all these warnings? It would be great if someone could figure out what the post-build package check is doing and why it isn't working for Mateusz. Simon == Start post-testsuite package check Timestamp Mon Jan 6 17:45:05 GMT 2014 for /5playpen/simonpj/HEAD-2/inplace/lib/package.conf.d/package.cache Timestamp Mon Jan 6 17:45:05 GMT 2014 for /5playpen/simonpj/HEAD-2/inplace/lib/package.conf.d (same as cache) using cache: /5playpen/simonpj/HEAD-2/inplace/lib/package.conf.d/package.cache Warning: haddock-interfaces: /5playpen/simonpj/HEAD-2/libraries/dph/dph-lifted-vseg/dist-install/doc/html/dph-lifted-vseg/dph-lifted-vseg.haddock doesn't exist or isn't a file Warning: haddock-interfaces: /5playpen/simonpj/HEAD-2/libraries/dph/dph-lifted-copy/dist-install/doc/html/dph-lifted-copy/dph-lifted-copy.haddock doesn't exist or isn't a file Warning: haddock-interfaces: /5playpen/simonpj/HEAD-2/libraries/dph/dph-lifted-boxed/dist-install/doc/html/dph-lifted-boxed/dph-lifted-boxed.haddock doesn't exist or isn't a file ...etc | -Original Message- | From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of | Mateusz Kowalczyk | Sent: 07 January 2014 03:13 | To: ghc-devs@haskell.org | Subject: Re: Validating with Haddock | | On 28/12/13 16:53, Mateusz Kowalczyk wrote: | Greetings, | | I'm trying to validate HEAD and I care that Haddock is built alongside | it (so --no-haddock is not an option). I get the following errors | listed at the bottom of this e-mail. How can I validate so that it all | builds? | | From what I understand, to validate I should: | * Have a stable compiler in my PATH (7.6.3) | * go to top level directory | * run 'sh validate' | | Am I missing steps? | | == Start post-build package check | Timestamp 2013-12-28 05:00:55 UTC for | /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d/package.cache | Timestamp 2013-12-28 05:00:55 UTC for | /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d (same as | cache) using cache: | /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d/package.cache | Timestamp 2013-12-28 05:22:27 UTC for | /home/shana/programming/ghc/inplace/lib/package.conf.d/package.cache | Timestamp 2013-12-28 05:22:27 UTC for | /home/shana/programming/ghc/inplace/lib/package.conf.d (same as cache) | using cache: | /home/shana/programming/ghc/inplace/lib/package.conf.d/package.cache | There are problems in package xhtml-3000.2.1: |dependency base-4.7.0.0-578628bf142f9304d05ce5581b5f8d76 doesn't | exist There are problems in package ghc-paths-0.1.0.9: |dependency base-4.7.0.0-578628bf142f9304d05ce5581b5f8d76 doesn't | exist | | The following packages are broken, either because they have a problem | listed above, or because they depend on a broken package. | xhtml-3000.2.1 | ghc-paths-0.1.0.9 | | | Ping. I need GHC to validate. Here's what I'm trying to achieve: as you | might know, I worked on Haddock over summer, rewriting the whole parser, | adding tests, fixing bugs, adding features. As Haddock ships with GHC | however (and is technically a GHC HQ package), we can not merge it | without making sure that GHC can build and validate with the changes. | | This has been a problem for me and Simon Hengel for quite a while. We | now have a branch with preliminary changes on | https://github.com/sol/haddock/tree/new-parser . We can not even begin | to try to merge the new features if the parser they are built upon is | not merged. With the recent calls to push out a 7.8 release candidate, I | think we're running out of time to get this in (or is it too late | already?). It is not the first time we've been asking for help here! | | Can someone say what are the steps I should take to get an OK from the | GHC HQ that we can push new-parser onto master? If we miss 7.8, the next | opportunity will be 7.10, because to get a new Haddock version you also | need a new compiler, which people only get during stable releases. | There's still a lot of work to be done on Haddock and I think it's | understandable that I don't want to do work on what effectively is an | 'outdated version'. I'm fine with changes being rejected because they | are deemed not good enough for some specific reason, but I'd hate the | changes to not make it because I can't get a confirmation from GHC HQ | that it's safe to do so. | | Thanks, hope to hear from someone soon. | | -- | Mateusz K. | ___ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
RE: Validating with Haddock
| Ping. I need GHC to validate. Here's what I'm trying to achieve: as you | might know, I worked on Haddock over summer, rewriting the whole parser, | adding tests, fixing bugs, adding features. As Haddock ships with GHC | however (and is technically a GHC HQ package), we can not merge it | without making sure that GHC can build and validate with the changes. | | This has been a problem for me and Simon Hengel for quite a while. We | now have a branch with preliminary changes on | https://github.com/sol/haddock/tree/new-parser . We can not even begin | to try to merge the new features if the parser they are built upon is | not merged. With the recent calls to push out a 7.8 release candidate, I | think we're running out of time to get this in (or is it too late | already?). It is not the first time we've been asking for help here! Actually I didn't know that you were asking to get something into 7.8. Haddock is maintained by David Waern and Simon Marlow, so the question of when to merge your changes into the main Haddock HEAD is up to them, not GHC HQ. We'll simply ship whatever Haddock we have when we cut the release candidate. (I know there is still some fuzz about when that will be; Austin is figuring that out now.) I'm copying David and Simon. Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 07/01/14 09:41, Simon Peyton Jones wrote: | Ping. I need GHC to validate. Here's what I'm trying to achieve: as you | might know, I worked on Haddock over summer, rewriting the whole parser, | adding tests, fixing bugs, adding features. As Haddock ships with GHC | however (and is technically a GHC HQ package), we can not merge it | without making sure that GHC can build and validate with the changes. | | This has been a problem for me and Simon Hengel for quite a while. We | now have a branch with preliminary changes on | https://github.com/sol/haddock/tree/new-parser . We can not even begin | to try to merge the new features if the parser they are built upon is | not merged. With the recent calls to push out a 7.8 release candidate, I | think we're running out of time to get this in (or is it too late | already?). It is not the first time we've been asking for help here! Actually I didn't know that you were asking to get something into 7.8. Haddock is maintained by David Waern and Simon Marlow, so the question of when to merge your changes into the main Haddock HEAD is up to them, not GHC HQ. We'll simply ship whatever Haddock we have when we cut the release candidate. (I know there is still some fuzz about when that will be; Austin is figuring that out now.) I'm copying David and Simon. Simon David stepped down and Simon Marlow has a long time ago too! It is now Simon Hengel who maintains it. The issue is that Simon could not get the tree to validate properly on his machine either so I'm here seeking help so that we can push up with confidence. Simon has e-mailed Austin on 22/11/2013 about it but we have been unable to verify that all is fine. Is it by now too late for 7.8? I'm afraid Simon H is away without much access to technology until the 20th. Un-CC'ing Simon M and David W and CC'ing Simon H. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
RE: Validating with Haddock
| David stepped down and Simon Marlow has a long time ago too! It is now | Simon Hengel who maintains it. OK, well perhaps you can immediately push a change to haddock.cabal to reflect this? That's how we know. | Is it by now too late for 7.8? I'm afraid Simon H is away without much | access to technology until the 20th. Realistically that would push 7.8 RC to the end of Jan. Why would that be better than pushing to head just after the 7.8 release? Will 7.8 users see a big improvement if it was in? What do others think? S ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 07/01/14 10:20, Simon Peyton Jones wrote: | David stepped down and Simon Marlow has a long time ago too! It is now | Simon Hengel who maintains it. OK, well perhaps you can immediately push a change to haddock.cabal to reflect this? That's how we know. I will try later but I think I don't have permissions. I can at best push to Simon's branch where he would periodically push to the GHC hosted repository (or perhaps it would get pulled from, I do not know). | Is it by now too late for 7.8? I'm afraid Simon H is away without much | access to technology until the 20th. Realistically that would push 7.8 RC to the end of Jan. Why would that be better than pushing to head just after the 7.8 release? Will 7.8 users see a big improvement if it was in? What do others think? The changes were mostly there for user benefit. The markup can now be escaped much better. If we can validate what's on Simon's new-parser branch reasonably quickly, we might even be able to push in the new features: new mark up, nested paragraphs, better lists, headers… I'm trying to push for 7.8 because Haddock ships with GHC and 7.8 is the stable release that everyone will be using in couple of months time. If the changes don't get into 7.8, we'll have to wait for the next stable release for the users to benefit. Is this incorrect? I was always under the impression that the only Haddock releases we can reasonably make are with stable GHC releases. Of course, anyone can compile HEAD and generate the docs for their own viewing but for example, Hackage will run stable compiler and all the docs will still be using old Haddock. I'd love to hear that I'm wrong about this and that Haddock releases separate from GHC are possible but I don't think that's the case. S -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
Hi Mateusz, I remember your email and I believe I responded with the OK at the time - my impression was that it was ready to be merged and would shortly be done after that, but I didn't hear anything back about it. I apologize for my dropping the ball. As for your actual error - ghc-paths is only used in Haddock when it's not built in the GHC tree (as per the cabal file,) so I find it very suspicious that your package check is mentioning it at all (it's not mentioned anywhere else in any GHC sources.) Can you verify that it's there with `./inplace/bin/ghc-pkg list`? I'm not even sure how it could possibly get involved. Finally, can you be more specific about exactly how you tested these changes with your modified Haddock? I presume it was something like: $ ... clone ghc source ... $ cd ghc $ ... get extra stuff with ./sync-all ... $ cd utils/haddock $ ... use git to grab your code from github ... $ cd ../.. $ sh ./validate But I'd like to make sure I know exactly what's going on. I can try testing your branch later today. On Tue, Jan 7, 2014 at 4:29 AM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 07/01/14 10:20, Simon Peyton Jones wrote: | David stepped down and Simon Marlow has a long time ago too! It is now | Simon Hengel who maintains it. OK, well perhaps you can immediately push a change to haddock.cabal to reflect this? That's how we know. I will try later but I think I don't have permissions. I can at best push to Simon's branch where he would periodically push to the GHC hosted repository (or perhaps it would get pulled from, I do not know). | Is it by now too late for 7.8? I'm afraid Simon H is away without much | access to technology until the 20th. Realistically that would push 7.8 RC to the end of Jan. Why would that be better than pushing to head just after the 7.8 release? Will 7.8 users see a big improvement if it was in? What do others think? The changes were mostly there for user benefit. The markup can now be escaped much better. If we can validate what's on Simon's new-parser branch reasonably quickly, we might even be able to push in the new features: new mark up, nested paragraphs, better lists, headers… I'm trying to push for 7.8 because Haddock ships with GHC and 7.8 is the stable release that everyone will be using in couple of months time. If the changes don't get into 7.8, we'll have to wait for the next stable release for the users to benefit. Is this incorrect? I was always under the impression that the only Haddock releases we can reasonably make are with stable GHC releases. Of course, anyone can compile HEAD and generate the docs for their own viewing but for example, Hackage will run stable compiler and all the docs will still be using old Haddock. I'd love to hear that I'm wrong about this and that Haddock releases separate from GHC are possible but I don't think that's the case. S -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 07/01/14 13:57, Simon Hengel wrote: Hey! Sorry for not being of much help with this right now. Regarding Haddock releases I think we updated the version used for Hackage independently of ghc before. Cheers. Oh, if that's the case then I no longer feel that it's urgent that we get it into 7.8 considering Hackage is where the bulk of the docs are. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
Yes, the skipped tests are normal. The testsuite has a concept of tests being built a certain 'way' - for example, you might test a piece of code by making sure it works compiled with -threaded, non-threaded, profiling, the LLVM backend, or any combination of those, etc. So a single *test* gives rise to multiple *test cases*. When you run validate, it runs it in a 'fast' mode by default as opposed to the slow mode. The fast mode only runs a subset of the overall test cases - it runs the most basic tests per file, which generally gives a pretty good indication as to what is going on. Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail. In any case, these results seem to indicate your branch looks quite OK, so I can try to merge this soon, if you think it is actually complete and ready. On Tue, Jan 7, 2014 at 12:13 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 07/01/14 14:42, Austin Seipp wrote: Hi Mateusz, I remember your email and I believe I responded with the OK at the time - my impression was that it was ready to be merged and would shortly be done after that, but I didn't hear anything back about it. I apologize for my dropping the ball. We contacted you because we thought it wouldn't be this much trouble to get an OK from the validate process. The code was technically more or less ready months ago, although Simon has been making some changes here and there. As for your actual error - ghc-paths is only used in Haddock when it's not built in the GHC tree (as per the cabal file,) so I find it very suspicious that your package check is mentioning it at all (it's not mentioned anywhere else in any GHC sources.) Can you verify that it's there with `./inplace/bin/ghc-pkg list`? I'm not even sure how it could possibly get involved. Finally, can you be more specific about exactly how you tested these changes with your modified Haddock? I presume it was something like: I had ran it on a as-is tree so that I could compare the results from before and after I put my changes in. I had just ran validate yesterday again (after sync-all) and I no longer get this package failure! $ ... clone ghc source ... $ cd ghc $ ... get extra stuff with ./sync-all ... $ cd utils/haddock $ ... use git to grab your code from github ... $ cd ../.. $ sh ./validate As I mention, it was on unchanged tree but this is how I'll do it when testing the changes. But I'd like to make sure I know exactly what's going on. I can try testing your branch later today. I think the original issue is now gone. I do get 8 unexpected failures and about 11000+ skipped tests! Is this normal? Should I be filing bugs? Should I create a separate thread? Can someone look at my log? You can download it from [1], it's about 8MB. If GHC itself compiles the test information into some files you'd prefer, please let me know, I simply redirected all output from the validate script into a log. [1]: http://fuuzetsu.co.uk/misc/validatelog -- Mateusz K. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 07/01/14 18:21, Austin Seipp wrote: Yes, the skipped tests are normal. The testsuite has a concept of tests being built a certain 'way' - for example, you might test a piece of code by making sure it works compiled with -threaded, non-threaded, profiling, the LLVM backend, or any combination of those, etc. So a single *test* gives rise to multiple *test cases*. When you run validate, it runs it in a 'fast' mode by default as opposed to the slow mode. The fast mode only runs a subset of the overall test cases - it runs the most basic tests per file, which generally gives a pretty good indication as to what is going on. Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail. In any case, these results seem to indicate your branch looks quite OK, so I can try to merge this soon, if you think it is actually complete and ready. These are the numbers from the clean tree. I will now merge in my changes, validate again, run Haddock test suite and let you know how it went. If I see similar results, I'll assume it's fine. I greatly appreciate the help I've been getting on this thread. @Simon H. Do you think that the new features could be merged fairly soon too, if the basic parser stuff checks out? Does anything extra need doing? -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On Tue, Jan 07, 2014 at 06:39:36PM +, Mateusz Kowalczyk wrote: On 07/01/14 18:21, Austin Seipp wrote: Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail. These are the numbers from the clean tree. The haddock perf numbers look pretty bad, especially the peak_megabytes_allocated: = haddock.base(normal) 429 of 3855 [0, 0, 0] peak_megabytes_allocated value is too high: Expectedpeak_megabytes_allocated: 139 +/-1% Actual peak_megabytes_allocated: 180 = haddock.Cabal(normal) 430 of 3855 [0, 1, 0] peak_megabytes_allocated value is too high: Expectedpeak_megabytes_allocated: 89 +/-1% Actual peak_megabytes_allocated: 150 = haddock.compiler(normal) 431 of 3855 [0, 2, 0] max_bytes_used value is too high: Expectedpeak_megabytes_allocated: 663 +/-1% Actual peak_megabytes_allocated: 794 I think it would be worth working out what's going on before merging more haddock changes. Thanks Ian ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 07/01/14 20:15, Ian Lynagh wrote: On Tue, Jan 07, 2014 at 06:39:36PM +, Mateusz Kowalczyk wrote: On 07/01/14 18:21, Austin Seipp wrote: Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail. These are the numbers from the clean tree. The haddock perf numbers look pretty bad, especially the peak_megabytes_allocated: = haddock.base(normal) 429 of 3855 [0, 0, 0] peak_megabytes_allocated value is too high: Expectedpeak_megabytes_allocated: 139 +/-1% Actual peak_megabytes_allocated: 180 = haddock.Cabal(normal) 430 of 3855 [0, 1, 0] peak_megabytes_allocated value is too high: Expectedpeak_megabytes_allocated: 89 +/-1% Actual peak_megabytes_allocated: 150 = haddock.compiler(normal) 431 of 3855 [0, 2, 0] max_bytes_used value is too high: Expectedpeak_megabytes_allocated: 663 +/-1% Actual peak_megabytes_allocated: 794 I think it would be worth working out what's going on before merging more haddock changes. Thanks Ian Hi Ian, Is there any guidance on how these tests are performed? More importantly, is there any log of how the performance changed over time? Is it Haddock's fault that it has become slower or is it the cause of GHC changes? PS: If there's no performance over time log, it might be worth introducing something! -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
For the record and other people reading - after a quick discussion on IRC, it simply looks like the 32-bit peak_megabytes_allocated numbers for those tests probably weren't updated at the same time as the 64bit ones, leaving them out of date. On Tue, Jan 7, 2014 at 3:07 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 07/01/14 20:15, Ian Lynagh wrote: On Tue, Jan 07, 2014 at 06:39:36PM +, Mateusz Kowalczyk wrote: On 07/01/14 18:21, Austin Seipp wrote: Also, the performance failures you're seeing are (I speculate) due to out of date performance numbers. Sometimes these numbers go up or down just due to code churn, but they're sometimes finnicky, because they may depend on the exact time a major GC happens or something. So a small wibble can cause them to sometimes occasionally fail. These are the numbers from the clean tree. The haddock perf numbers look pretty bad, especially the peak_megabytes_allocated: = haddock.base(normal) 429 of 3855 [0, 0, 0] peak_megabytes_allocated value is too high: Expectedpeak_megabytes_allocated: 139 +/-1% Actual peak_megabytes_allocated: 180 = haddock.Cabal(normal) 430 of 3855 [0, 1, 0] peak_megabytes_allocated value is too high: Expectedpeak_megabytes_allocated: 89 +/-1% Actual peak_megabytes_allocated: 150 = haddock.compiler(normal) 431 of 3855 [0, 2, 0] max_bytes_used value is too high: Expectedpeak_megabytes_allocated: 663 +/-1% Actual peak_megabytes_allocated: 794 I think it would be worth working out what's going on before merging more haddock changes. Thanks Ian Hi Ian, Is there any guidance on how these tests are performed? More importantly, is there any log of how the performance changed over time? Is it Haddock's fault that it has become slower or is it the cause of GHC changes? PS: If there's no performance over time log, it might be worth introducing something! -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
Excellent, thank you. We should really fix the 32bit performance numbers too I think, based on what we discussed on IRC earlier. Would you like to submit a patch for that too please? You can find the numbers in testsuite/tests/perf/haddock/all.T. Also, is there any new documentation we should need for this? Is all the new stuff properly documented somewhere? Etc. On Tue, Jan 7, 2014 at 6:20 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 07/01/14 21:20, Austin Seipp wrote: For the record and other people reading - after a quick discussion on IRC, it simply looks like the 32-bit peak_megabytes_allocated numbers for those tests probably weren't updated at the same time as the 64bit ones, leaving them out of date. I have now validated GHC with the new Haddock stuff in place. You can see the new log at [1]. The end result is the same as validation on a tree without changes: same 8 tests failing. I have also built and ran Haddock's own tests with HEAD and they now all check out. The branch at [2] should now be ready to be merged into upstream Haddock. If someone could merge that in, that'd be great. This is the new parser which contains few bug fixes. We have more changes than this which include user-visible features and new documentation. I'll prepare and validate those for you tomorrow and bother you some more. Let me know if anything needs changing. Thanks! [1]: http://fuuzetsu.co.uk/misc/validateloghaddock [2]: https://github.com/sol/haddock/tree/new-parser -- Mateusz K. -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
On 07/01/14 04:17, Carter Schonwald wrote: Well said points. 1) perhaps opening a ticket on ghc trac for your problem is a good next step. That way folks who are better at reading trac than email can help! I'll do so tomorrow if I don't get any replies with tips. 2) if the pattern synonyms branch gets merged in, you'll have to upstream the associated changes to haddock too right? It's not a show-stopper if Haddock can't document something. In fact there are many things it can't document already (GADT type constructors are an easy one). If someone writes the pattern synonym stuff for existing Haddock it's not a problem. The proposed changes from new-parser don't touch the parts that pattern synonyms would and if they did, it'd be easy to merge. Usually GHC HQ folk patch up Haddock when they change API so that it can still compile but everything extra tends to be a ‘if we can get it to document the new bleeding-edge feature, then great, if not, someone will make a ticket later’. I actually attempted to make Haddock work with some extra stuff that it currently can't document but because it so heavily depends on GHC, I need my GHC tree validating for that too. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
i'd really recommend asking on #ghc and filing a ticket on trac preemptively. Different people reply better on different channels On Tue, Jan 7, 2014 at 12:23 AM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.ukwrote: On 07/01/14 04:17, Carter Schonwald wrote: Well said points. 1) perhaps opening a ticket on ghc trac for your problem is a good next step. That way folks who are better at reading trac than email can help! I'll do so tomorrow if I don't get any replies with tips. 2) if the pattern synonyms branch gets merged in, you'll have to upstream the associated changes to haddock too right? It's not a show-stopper if Haddock can't document something. In fact there are many things it can't document already (GADT type constructors are an easy one). If someone writes the pattern synonym stuff for existing Haddock it's not a problem. The proposed changes from new-parser don't touch the parts that pattern synonyms would and if they did, it'd be easy to merge. Usually GHC HQ folk patch up Haddock when they change API so that it can still compile but everything extra tends to be a ‘if we can get it to document the new bleeding-edge feature, then great, if not, someone will make a ticket later’. I actually attempted to make Haddock work with some extra stuff that it currently can't document but because it so heavily depends on GHC, I need my GHC tree validating for that too. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Validating with Haddock
Greetings, I'm trying to validate HEAD and I care that Haddock is built alongside it (so --no-haddock is not an option). I get the following errors listed at the bottom of this e-mail. How can I validate so that it all builds? From what I understand, to validate I should: * Have a stable compiler in my PATH (7.6.3) * go to top level directory * run ‘sh validate’ Am I missing steps? == Start post-build package check Timestamp 2013-12-28 05:00:55 UTC for /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d/package.cache Timestamp 2013-12-28 05:00:55 UTC for /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d (same as cache) using cache: /home/shana/.ghc/i386-linux-7.7.20131227/package.conf.d/package.cache Timestamp 2013-12-28 05:22:27 UTC for /home/shana/programming/ghc/inplace/lib/package.conf.d/package.cache Timestamp 2013-12-28 05:22:27 UTC for /home/shana/programming/ghc/inplace/lib/package.conf.d (same as cache) using cache: /home/shana/programming/ghc/inplace/lib/package.conf.d/package.cache There are problems in package xhtml-3000.2.1: dependency base-4.7.0.0-578628bf142f9304d05ce5581b5f8d76 doesn't exist There are problems in package ghc-paths-0.1.0.9: dependency base-4.7.0.0-578628bf142f9304d05ce5581b5f8d76 doesn't exist The following packages are broken, either because they have a problem listed above, or because they depend on a broken package. xhtml-3000.2.1 ghc-paths-0.1.0.9 -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
