Re: Starting GHC development.
Does anyone actually know which tests are supposed to fail on 'validate'? On Fri, Jan 3, 2014 at 10:43 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 03/01/14 13:27, Simon Peyton-Jones wrote: [snip] Thank you. We need lots of help! [snip] While I hate to interrupt this thread, I think this is a good chance to mention something. I think the big issue for joining GHC development is the lack of communication on the mailing list. There are many topics where a person has a problem with GHC tree (can't validate/build, some tests are failing), posts to GHC devs seeking help and never gets a reply. This is very discouraging and often makes it outright impossible to contribute. An easy example is the failing tests one: unfortunately some tests are known to fail, but they are only known to fail to existing GHC devs. A new person tries to validate clean tree, gets test failures, asks for help on GHC devs, doesn't get any, gives up. Is there any better way to get through than ghc-devs? Even myself I'd love to get started but if I can't get help even getting the ‘clean’ tree to a state where I'm confident it's not a problem with my machine, how am I to write patches for anything? A more serious example is that the work I did over summer on Haddock still hasn't been pushed in. Why? Because neither Simon Hengel nor myself can ensure that we haven't broken anything as neither of use gets a clean validate. I have in fact asked for help recently with this but to no avail and I do know Simon also sought help in the past to no avail. I have also tried to join the development quite a few months in the past now but due to failing tests on validate and lack of help, I had to give up on that. Please guys, try to increase responsiveness to posts on this list. It's very easy to scroll down in your mail client and see just how many threads never got a single reply. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Sincerely yours, -- Daniil ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Starting GHC development.
On 12/01/14 10:25, Daniil Frumin wrote: Does anyone actually know which tests are supposed to fail on 'validate'? AFAIK the official stance is that you should see 0 failures. Unofficially it seems that there's leniency and the tree seems to be in a state with few tests failing consistently. It might be just my machine though, but whenever I post my build logs, there seems to be no sense of urgency to investigate so it does not seem like anyone cares or the issue is known/being worked on. Unfortunately, the side effect of this (and what put me off when I tried to write some stuff for GHC months ago) was that a new developer comes, tries to build clean tree and it fails. It's pretty discouraging. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Starting GHC development.
When you see tests failing with validate I's suggest going to #ghc IRC channel and asking there. You're most likely to get an up-to-date answer. Basically, when you're working on some changes in GHC I'd suggest to run ./validate on the master branch to see what (if any) tests are failing. When validating your changes you'll have to see whether your modifications introduced any new failures or not (or maybe fixed the existing ones). In any case I strongly suggest running the failing tests (both on master branch and on yours) to make sure that they fail in the same way. Janek Dnia niedziela, 12 stycznia 2014, Daniil Frumin napisał: Does anyone actually know which tests are supposed to fail on 'validate'? On Fri, Jan 3, 2014 at 10:43 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 03/01/14 13:27, Simon Peyton-Jones wrote: [snip] Thank you. We need lots of help! [snip] While I hate to interrupt this thread, I think this is a good chance to mention something. I think the big issue for joining GHC development is the lack of communication on the mailing list. There are many topics where a person has a problem with GHC tree (can't validate/build, some tests are failing), posts to GHC devs seeking help and never gets a reply. This is very discouraging and often makes it outright impossible to contribute. An easy example is the failing tests one: unfortunately some tests are known to fail, but they are only known to fail to existing GHC devs. A new person tries to validate clean tree, gets test failures, asks for help on GHC devs, doesn't get any, gives up. Is there any better way to get through than ghc-devs? Even myself I'd love to get started but if I can't get help even getting the ‘clean’ tree to a state where I'm confident it's not a problem with my machine, how am I to write patches for anything? A more serious example is that the work I did over summer on Haddock still hasn't been pushed in. Why? Because neither Simon Hengel nor myself can ensure that we haven't broken anything as neither of use gets a clean validate. I have in fact asked for help recently with this but to no avail and I do know Simon also sought help in the past to no avail. I have also tried to join the development quite a few months in the past now but due to failing tests on validate and lack of help, I had to give up on that. Please guys, try to increase responsiveness to posts on this list. It's very easy to scroll down in your mail client and see just how many threads never got a single reply. -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validating with Haddock
Hi Mateusz, I've pushed your work and tweaked the testsuite performance numbers on 64bit. The 32bit ones are out of date, but I'll fix them shortly. I also fixed some of the documentation errors. Thanks for all your hard work. On Fri, Jan 10, 2014 at 4:06 PM, Mateusz Kowalczyk fuuze...@fuuzetsu.co.uk wrote: On 10/01/14 10:01, Mateusz Kowalczyk wrote: Hi all, I have now merged in the new parser and new features onto a single branch. I'm having some issues validating with HEAD at the moment (#8661, unrelated problem) but while I get that sorted out, someone might want to try validating with Haddock changes on their own platform. The full branch is at [1]. I have squashed the changes to what I feel is the minimum number of commits until they completely stop making sense. It should apply cleanly on top of current Haddock master branch. The documentation is updated so you can read about what changed. Feel free to ask any questions. I will post again once I can confirm that the branch validates for me without any new test failures. Thanks for your patience. [1]: https://github.com/Fuuzetsu/haddock/tree/new-features This is just a simple follow up to say that the changes don't seem to break anything new on 32-bit Linux. I provide my validate logs before[1] and after[2] Haddock changes. Here's a word of warning: previously, when the mark-up wasn't 100% clear, we'd get a parse error and no documentation for the whole package. The new parser no longer does this and instead does its best to parse and present everything. This means that any Haddock parse failures should be reported as bugs. As you can see in [1], there were some parse failures in the past (look for ‘doc comment parse failed’) and they will now be rendered. This means the documentation might look bad in those places so it's probably worth while visiting those places and having a look. On an upside, at least we now have documentation for those packages. Validation was ran on commit 15a3de1288fe9d055f3dc92d554cb59b3528fa30 including #8661 fixes. Here's the relevant tail of the logs: Unexpected results from: TEST=lazy-bs-alloc T1969 T3064 T4801 T3294 T5498 haddock.Cabal haddock.compiler haddock.base OVERALL SUMMARY for test run started at Fri Jan 10 12:45:39 2014 GMT 0:17:23 spent to go through 3861 total tests, which gave rise to 15072 test cases, of which 11547 were skipped 28 had missing libraries 3432 expected passes 56 expected failures 0 caused framework failures 0 unexpected passes 9 unexpected failures Unexpected failures: deriving/should_fail T5498 [stderr mismatch] (normal) perf/compiler T1969 [stat too good] (normal) perf/compiler T3064 [stat not good enough] (normal) perf/compiler T3294 [stat not good enough] (normal) perf/compiler T4801 [stat not good enough] (normal) perf/haddock haddock.Cabal [stat not good enough] (normal) perf/haddock haddock.base [stat not good enough] (normal) perf/haddock haddock.compiler [stat not good enough] (normal) perf/should_run lazy-bs-alloc [stat too good] (normal) gmake[2]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' gmake[1]: Leaving directory `/home/shana/programming/ghc/testsuite/tests' == Start post-testsuite package check Timestamp 2014-01-10 12:45:36.897842164 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache Timestamp 2014-01-10 12:45:36 UTC for /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d (older than cache) using cache: /home/shana/programming/ghc/bindisttest/install dir/lib/ghc-7.7.20140109/package.conf.d/package.cache == End post-testsuite package check --- Oops! Looks like you have some unexpected test results or framework failures. Please fix them before pushing/sending patches. --- The failures are the same in both logs. Thanks! [1]: http://fuuzetsu.co.uk/misc/segfix [2]: http://fuuzetsu.co.uk/misc/segfixhaddock -- Mateusz K. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Regards, Austin Seipp, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
[PATCH 2/2] fix binary linking errors on Solaris due to misplacing of -Wl, -u, symbol option
--- compiler/main/DriverPipeline.hs | 11 ++- 1 files changed, 10 insertions(+), 1 deletions(-) diff --git a/compiler/main/DriverPipeline.hs b/compiler/main/DriverPipeline.hs index 337778e..1c593b6 100644 --- a/compiler/main/DriverPipeline.hs +++ b/compiler/main/DriverPipeline.hs @@ -1790,7 +1790,16 @@ linkBinary' staticLink dflags o_files dep_packages = do -- HS packages, because libtool doesn't accept other options. -- In the case of iOS these need to be added by hand to the -- final link in Xcode. -else package_hs_libs ++ extra_libs ++ other_flags +else other_flags ++ package_hs_libs ++ extra_libs -- -Wl,-u,sym contained in other_flags + -- needs to be put before -lpackage, + -- otherwise Solaris linker fails linking + -- a binary with unresolved symbols in RTS + -- which are defined in base package + -- the reason for this is a note in ld(1) about + -- '-u' option: The placement of this option + -- on the command line is significant. + -- This option must be placed before the library + -- that defines the symbol. pkg_framework_path_opts - if platformUsesFrameworks platform -- 1.7.3.2 ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
[PATCH 1/2] add handling of Solaris linker into SysTools
--- compiler/main/DynFlags.hs |1 + compiler/main/SysTools.lhs |9 + 2 files changed, 10 insertions(+), 0 deletions(-) diff --git a/compiler/main/DynFlags.hs b/compiler/main/DynFlags.hs index 70d2a81..e253bae 100644 --- a/compiler/main/DynFlags.hs +++ b/compiler/main/DynFlags.hs @@ -3721,6 +3721,7 @@ data LinkerInfo = GnuLD[Option] | GnuGold [Option] | DarwinLD [Option] + | SolarisLD [Option] | UnknownLD deriving Eq diff --git a/compiler/main/SysTools.lhs b/compiler/main/SysTools.lhs index 46f8a86..0c86c18 100644 --- a/compiler/main/SysTools.lhs +++ b/compiler/main/SysTools.lhs @@ -638,6 +638,7 @@ neededLinkArgs :: LinkerInfo - [Option] neededLinkArgs (GnuLD o) = o neededLinkArgs (GnuGold o) = o neededLinkArgs (DarwinLD o) = o +neededLinkArgs (SolarisLD o) = o neededLinkArgs UnknownLD = [] -- Grab linker info and cache it in DynFlags. @@ -676,6 +677,14 @@ getLinkerInfo' dflags = do -- Process the executable call info - catchIO (do case os of + OSSolaris2 - + -- Solaris uses its own Solaris linker. Even all + -- GNU C are receommended to configure with Solaris + -- linker instead of using GNU binutils linker. Also + -- all GCC distributed with Solaris follows this rule + -- precisely so we assume here, the Solaris linker is + -- used. + return $ SolarisLD [] OSDarwin - -- Darwin has neither GNU Gold or GNU LD, but a strange linker -- that doesn't support --version. We can just assume that's -- 1.7.3.2 ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Enable TypeHoles by default?
Hello, I propose to enable -XTypeHoles in GHC by default. Unlike other -X* flags, holes do not really change meaning of the program, they only change error messages. Instead of _x not in scope, we effectively get _x not in scope, its expected type is a - a. You get it only if you precede the identifier not in scope with underscore, so in some sense you declare the intention of using holes. Two possible issues: (a) If you use -fdefer-type-errors, then a program might compile, while previously it did not. However, we should facilitate compiling with defer-type-errors, so I don't think this is a disadvantage. (b) The identifier _ becomes both a pattern and a hole by default, which might confuse new users. Reply: I have never seen anyone ask why code such as Just _ - _ does not work. IMO the productivity boost by having holes by default outweighs those two objections. I am open to hearing any other possible issues others might find. The change is trivial implementation-wise; add Opt_TypeHoles to the list in languageExtensions Nothing in DynFlags. -KG ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: panic when compiling SHA
On 10/01/2014, at 6:17 , Adam Wick aw...@galois.com wrote: That’s the problem with SHA, then. The implementation (and the spec, really) is essentially a long combination of the form: let x_n5 = small_computation x_n1 x_n2 x_n3 x_n4 x_n6 = small_computation x_n2 x_n3 x_n4 x_n5 … Which has ~70 entries. The actual number of live variables alive at any time should be relatively small, but if slots aren’t getting reused there’s going to be some significant blowup. (To be honest, I had figured — and thought I had validated — that doing it this way would give the compiler the best chance at generating optimal code, but it appears I merely set myself up to hit this limitation several years later.) If this [1] is the current version then I don't think there is any performance reason to manually unroll the loops like that. If you write a standard tail-recursive loop then the branch predictor in the processor should make the correct prediction for all iterations except the last one. You'll get one pipeline stall at the end due to a mis-predicted backward branch, but it won't matter in terms of absolute percentage of execution time. You generally only need to worry about branches if the branch flag flips between True and False frequently. If you care deeply about performance then on some processors it can be helpful to unfold this sort of code so that the SHA constants are represented as literals in the instruction stream instead of in static data memory -- but that ability is very processor specific and you'd need to really stare at the emitted assembly code to see if it's worthwhile. Ben. [1] https://github.com/GaloisInc/SHA/blob/master/Data/Digest/Pure/SHA.hs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Enable TypeHoles by default?
So would this *improve* error message quality for new users? Defaults that make it easier for haskellers old and new both are a tough balance to make! On Sun, Jan 12, 2014 at 6:40 PM, Dan Frumin difru...@gmail.com wrote: Hi! On 13 Jan 2014, at 02:56, Krzysztof Gogolewski krz.gogolew...@gmail.com wrote: Hello, I propose to enable -XTypeHoles in GHC by default. Unlike other -X* flags, holes do not really change meaning of the program, they only change error messages. Instead of _x not in scope, we effectively get _x not in scope, its expected type is a - a. You get it only if you precede the identifier not in scope with underscore, so in some sense you declare the intention of using holes. Two possible issues: (a) If you use -fdefer-type-errors, then a program might compile, while previously it did not. However, we should facilitate compiling with defer-type-errors, so I don't think this is a disadvantage. (b) The identifier _ becomes both a pattern and a hole by default, which might confuse new users. Reply: I have never seen anyone ask why code such as Just _ - _ does not work. I do think that having _ both as a pattern and a hole might be confusing, I can see that. However that's more of a syntax issue, than an issue about default extensions IMO IMO the productivity boost by having holes by default outweighs those two objections. I am open to hearing any other possible issues others might find. The change is trivial implementation-wise; add Opt_TypeHoles to the list in languageExtensions Nothing in DynFlags. -KG ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: panic when compiling SHA
agreed, this level of unrolling would cause problems even in most C compilers! When I write unrolled simd C code, I use a fixed number of variables that corresponds to the # of registers that are live on my target arch (yes, internally the turn into SSA which then does an ANF/CPS style and such), but by shadowing/resuing a fixed number of names, i can make it syntactically clear what the lifetimes of my variables is. But yeah, branch predictors are pretty good on modern hardware, a loop is worth considering. On Sun, Jan 12, 2014 at 10:06 PM, Ben Lippmeier b...@ouroborus.net wrote: On 10/01/2014, at 6:17 , Adam Wick aw...@galois.com wrote: That’s the problem with SHA, then. The implementation (and the spec, really) is essentially a long combination of the form: let x_n5 = small_computation x_n1 x_n2 x_n3 x_n4 x_n6 = small_computation x_n2 x_n3 x_n4 x_n5 … Which has ~70 entries. The actual number of live variables alive at any time should be relatively small, but if slots aren’t getting reused there’s going to be some significant blowup. (To be honest, I had figured — and thought I had validated — that doing it this way would give the compiler the best chance at generating optimal code, but it appears I merely set myself up to hit this limitation several years later.) If this [1] is the current version then I don't think there is any performance reason to manually unroll the loops like that. If you write a standard tail-recursive loop then the branch predictor in the processor should make the correct prediction for all iterations except the last one. You'll get one pipeline stall at the end due to a mis-predicted backward branch, but it won't matter in terms of absolute percentage of execution time. You generally only need to worry about branches if the branch flag flips between True and False frequently. If you care deeply about performance then on some processors it can be helpful to unfold this sort of code so that the SHA constants are represented as literals in the instruction stream instead of in static data memory -- but that ability is very processor specific and you'd need to really stare at the emitted assembly code to see if it's worthwhile. Ben. [1] https://github.com/GaloisInc/SHA/blob/master/Data/Digest/Pure/SHA.hs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
