Re: GHC's Fall 2018 HCAR submission

[Ben Gamari]

Artem Pelenitsyn  writes:

> Ben,
>
> I assume, writing a Pandoc writer for TracWiki shouldn't be that hard.
> Would that be of any help?
>
Well, there would need to be both a reader and a writer, since we
ultimately need to end up with TeX. Indeed having these would help
immensely.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

The future of Phabricator

[Ben Gamari]

TL;DR. For several reasons I think we should consider alternatives to
   Phabricator. My view is that GitLab seems like the best option.


Hello everyone,

Over the past year I have been growing increasingly weary of our
continued dependence on Phabricator. Without a doubt, its code review
interface is the best I have used. However, for a myriad of reasons I
am recently questioning whether it is still the best tool for GHC's
needs.


# The problem

There are a number of reasons why I am currently uncertain about
Phabricator.

For one, at this point we have no options for support in the event that
something goes wrong as the company responsible for Phabricator,
Phacility, has closed their support channels to non-paying customers.
Furthermore, in the past year or two Phacility has been placing their
development resources in the parts their customers pay them for, which
appear to be much different that the parts that we actively use. For
this reason, some parts that we rely on seem oddly half-finished.

This concern was recently underscored by some rather unfortunate
misfeatures in Harbormaster which resulted in broken CI after the
Hadrian merge and now apparent bugs which have made it difficult to
migrate to the CircleCI integration we previously prepared.

Perhaps most importantly, in our recent development priorities survey
our use of Phabricator was the most common complaint by a fair margin,
both in case of respondents who have contributed patches and those who
have not. On the whole, past contributors and potential future
contributors alike have strongly indicated that they want a more
Git-like experience. Of course, this wasn't terribly surprising; this
is just the most recent case where contributors have made this
preference known.

Frankly, in a sense it is hard to blame them. The fact that users need
to install a PHP tool, Arcanist, to contribute anything but
documentation patches has always seemed like unnecessary friction to me
and I would be quite happy to be rid of it. Indeed we have had a quite
healthy number of GitHub documentation patches since we started
accepting them. This makes me thing that there may indeed be potential
contributoes that we are leaving on the table.


# What to do

With Rackspace support ending at the end of year, now may be a good
time to consider whether we really want to continue on this road.
Phabricator is great at code review but I am less and less certain that
it is worth the maintenance uncertainty and potential lost contributors
that it costs.

Moreover, good alternatives seem closer at-hand than they were when we
deployed Phabricator.


## Move to GitHub

When people complain about our infrastructure, they often use GitHub as
the example of what they would like to see. However, realistically I
have a hard time seeing GitHub as a viable option. Its feature set is simply
insufficient enough to handle the needs of a larger project like GHC
without significant external tooling (as seen in the case of Rust-lang).

The concrete reasons have been well-documented in previous discussions
but, to summarize,

 * its review functionality is extremely painful to use with larger
   patches

 * rebased patches lead to extreme confusion and lost review comments

 * it lacks support for pre-receive hooks, which serve as a last line of
   defense to prevent unintended submodule changes

 * its inability to deal with external tickets is problematic

 * there is essentially no possibility that we could eventually migrate
   GHC's tickets to GitHub's ticket tracker without considerable data
   loss (much less manage the ticket load that would result), meaning
   that we would forever be stuck with maintaining Trac.

 * on a personal note, its search functionality has often left me
   empty-handed

On the whole, these issues seem pretty hard to surmount.


## Move to GitLab

In using GitLab for another project over the past months I have been
positively surprised by its quality. It handles rebased merge requests
far better than GitHub, has functional search, and quite a usable review
interface. Furthermore, upstream has been extremely responsive to
suggestions for improvement [1]. Even out-of-the-box it seems to be
flexible enough to accommodate our needs, supporting integration with
external issue trackers, having reasonable release management features,
and support for code owners to automate review triaging (replacing much
of the functionality of Phabricator's Herald).

Finally, other FOSS projects' [3] recent migrations from Phabrictor to
GitLab have shown that GitLab-the-company is quite willing to offer help
when needed. I took some time this weekend to try setting up a quick GHC
instance [2] to play around with. Even after just a few hours of playing
around I think the result is surprisingly usable.

Out of curiosity I also played around with importing some tickets from
Trac (building on Matt Pickering's Trac-to-Maniphest migration tool).
With relatively little effort I w

Re: GHC's Fall 2018 HCAR submission

[Artem Pelenitsyn]

Ben,

I assume, writing a Pandoc writer for TracWiki shouldn't be that hard.
Would that be of any help?

--
Best, Artem

On Mon, 29 Oct 2018 at 16:39 Ben Gamari  wrote:

> Artem Pelenitsyn  writes:
>
> > Hi Ben,
> >
> > I see. Have you considered using online converting tools like
> > try-pandoc[1]? Is it still painful?
> >
> I do use Pandoc to convert from Markdown to Latex. However, Pandoc does
> not support Trac's wiki syntax.
>
> Cheers,
>
> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC's Fall 2018 HCAR submission

[Ben Gamari]

Artem Pelenitsyn  writes:

> Hi Ben,
>
> I see. Have you considered using online converting tools like
> try-pandoc[1]? Is it still painful?
>
I do use Pandoc to convert from Markdown to Latex. However, Pandoc does
not support Trac's wiki syntax.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC's Fall 2018 HCAR submission

[Artem Pelenitsyn]

Hi Ben,

I see. Have you considered using online converting tools like
try-pandoc[1]? Is it still painful?

[1]: https://pandoc.org/try/

On Mon, 29 Oct 2018 at 15:32 Ben Gamari  wrote:

> Artem Pelenitsyn  writes:
>
> > On Mon, 29 Oct 2018 at 14:25 Evan Laforge  wrote:
> >
> >> Also, when I copy paste the links in the "At the time of writing"
> >> section, the backslashes in the search query mess it up.  Maybe a
> >> markdown rendering step would remove those?
> >>
> >
> > I'm not sure It makes much sense to use {{{...}}} to format the document
> in
> > the first place. Maybe use normal wiki-text? I can decorate links if this
> > is approved.
> >
> This is what I have done in the past. However, it is unfortunately quite
> labor intensive since I need to convert the document to Wiki markup,
> then back to TeX for submission.
>
> Instead with this iteration I have decided to try just writing the thing
> in Markdown and convert to TeX at the end. I was hoping to install a
> Trac processor for Markdown rendering but sadly Trac put up resistance.
>
> Cheers,
>
> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Using GHC Core as a Language Target

[Ara Adkins]

That’s a brilliant resource! Thanks so much for the links. 

_ara

> On 29 Oct 2018, at 19:11, Csaba Hruska  wrote:
> 
> There is also a nice intro blog post about GHC internals with an example how 
> to compile a custom constructed module AST.
> Dive into GHC: Pipeline
> Dive into GHC: Intermediate Forms
> Dive into GHC: Targeting Core
> Cheers,
> Csaba Hruska
> 
>> On Thu, Oct 25, 2018 at 8:51 PM Ara Adkins  wrote:
>> Heya,
>> 
>> Those are exactly the kind of pointers I was hoping for. Thanks Iavor. 
>> 
>> I’m sure I’ll have more questions with time, but that’s a great starting 
>> point. 
>> 
>> _ara
>> 
>>> On 25 Oct 2018, at 19:20, Iavor Diatchki  wrote:
>>> 
>>> Hello,
>>> 
>>> I have not done what you are asking, but here is how I'd approach the 
>>> problem.   
>>> 
>>> 1. Assuming you already have some Core, you'd have to figure out how to 
>>> include it with the rest of the GHC pipeline:
>>> * A lot of the code that glues everything together is in 
>>> `compiler/main`. Modules of interest seem to be `DriverPipeline`, 
>>> `HscMain`, and `PipelineMoand`
>>> * A quick looks suggests that maybe you want to call `hscGenHardCode` 
>>> in `HscMain`, with your core program inside the `CgGuts` argument.
>>> * Exactly how you setup things probably depends on how much of the rest 
>>> of the Haskell ecosystem you are trying to integrate with (separate 
>>> compilation, avoiding recompilation, support for packages, etc.)
>>> 
>>> 2. The syntax for Core is in `compiler/coreSyn`, with the basic AST being 
>>> in module `CoreSyn`.   Module `MkCore` has a lot of helpers for working 
>>> with core syntax.
>>> 
>>> 3. The "desugarer" (in `compiler/deSugar`) is the GHC phase that translates 
>>> the front end syntax (hsSyn) into core, so that should have lots of 
>>> examples of how to generate core.
>>> 
>>> Cheers,
>>> -Iavor
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
 On Mon, Oct 22, 2018 at 1:46 AM Ara Adkins  wrote:
 Hey All,
 
 I was chatting to SPJ about the possibility of using GHC Core + the rest 
 of the GHC compilation pipeline as a target for a functional language, and 
 he mentioned that asking here would likely be more productive when it 
 comes to the GHC API.
 
 I'm wondering where the best place would be for me to look in the API for 
 building core expressions, and also whether it is possible to trigger the 
 GHC code-generation pipeline from the core stage onwards.
 
 Best,
 Ara
 ___
 ghc-devs mailing list
 ghc-devs@haskell.org
 http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC's Fall 2018 HCAR submission

[Ben Gamari]

Artem Pelenitsyn  writes:

> On Mon, 29 Oct 2018 at 14:25 Evan Laforge  wrote:
>
>> Also, when I copy paste the links in the "At the time of writing"
>> section, the backslashes in the search query mess it up.  Maybe a
>> markdown rendering step would remove those?
>>
>
> I'm not sure It makes much sense to use {{{...}}} to format the document in
> the first place. Maybe use normal wiki-text? I can decorate links if this
> is approved.
>
This is what I have done in the past. However, it is unfortunately quite
labor intensive since I need to convert the document to Wiki markup,
then back to TeX for submission.

Instead with this iteration I have decided to try just writing the thing
in Markdown and convert to TeX at the end. I was hoping to install a
Trac processor for Markdown rendering but sadly Trac put up resistance.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Using GHC Core as a Language Target

[Csaba Hruska]

There is also a nice intro blog post about GHC internals with an example
how to compile a custom constructed module AST.

   - Dive into GHC: Pipeline 
   - Dive into GHC: Intermediate Forms
   
   - Dive into GHC: Targeting Core
   

Cheers,
Csaba Hruska

On Thu, Oct 25, 2018 at 8:51 PM Ara Adkins  wrote:

> Heya,
>
> Those are exactly the kind of pointers I was hoping for. Thanks Iavor.
>
> I’m sure I’ll have more questions with time, but that’s a great starting
> point.
>
> _ara
>
> On 25 Oct 2018, at 19:20, Iavor Diatchki  wrote:
>
> Hello,
>
> I have not done what you are asking, but here is how I'd approach the
> problem.
>
> 1. Assuming you already have some Core, you'd have to figure out how to
> include it with the rest of the GHC pipeline:
> * A lot of the code that glues everything together is in
> `compiler/main`. Modules of interest seem to be `DriverPipeline`,
> `HscMain`, and `PipelineMoand`
> * A quick looks suggests that maybe you want to call `hscGenHardCode`
> in `HscMain`, with your core program inside the `CgGuts` argument.
> * Exactly how you setup things probably depends on how much of the
> rest of the Haskell ecosystem you are trying to integrate with (separate
> compilation, avoiding recompilation, support for packages, etc.)
>
> 2. The syntax for Core is in `compiler/coreSyn`, with the basic AST being
> in module `CoreSyn`.   Module `MkCore` has a lot of helpers for working
> with core syntax.
>
> 3. The "desugarer" (in `compiler/deSugar`) is the GHC phase that
> translates the front end syntax (hsSyn) into core, so that should have lots
> of examples of how to generate core.
>
> Cheers,
> -Iavor
>
>
>
>
>
>
>
> On Mon, Oct 22, 2018 at 1:46 AM Ara Adkins  wrote:
>
>> Hey All,
>>
>> I was chatting to SPJ about the possibility of using GHC Core + the rest
>> of the GHC compilation pipeline as a target for a functional language, and
>> he mentioned that asking here would likely be more productive when it comes
>> to the GHC API.
>>
>> I'm wondering where the best place would be for me to look in the API for
>> building core expressions, and also whether it is possible to trigger the
>> GHC code-generation pipeline from the core stage onwards.
>>
>> Best,
>> Ara
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC's Fall 2018 HCAR submission

[Artem Pelenitsyn]

On Mon, 29 Oct 2018 at 14:25 Evan Laforge  wrote:

> Also, when I copy paste the links in the "At the time of writing"
> section, the backslashes in the search query mess it up.  Maybe a
> markdown rendering step would remove those?
>

I'm not sure It makes much sense to use {{{...}}} to format the document in
the first place. Maybe use normal wiki-text? I can decorate links if this
is approved.

--
Best, Artem


> On Sun, Oct 28, 2018 at 6:24 PM Ben Gamari  wrote:
> >
> > Hello everyone,
> >
> > The Haskell Community Activities Report is coming up and I have prepared
> > the skeleton for GHC's contribution [1]. If you have a project cooking
> > or have recently had a patch land do have a look to make sure it's
> > recognized in the submission.
> >
> > Cheers,
> >
> > - Ben
> >
> >
> > https://ghc.haskell.org/trac/ghc/wiki/Status/Oct18
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC's Fall 2018 HCAR submission

[Evan Laforge]

There are some incomplete sentences under "Further improvements to
runtime performance:"

Also, congratulations to Tamar Christina for becoming the new IO manager!

Also, when I copy paste the links in the "At the time of writing"
section, the backslashes in the search query mess it up.  Maybe a
markdown rendering step would remove those?
On Sun, Oct 28, 2018 at 6:24 PM Ben Gamari  wrote:
>
> Hello everyone,
>
> The Haskell Community Activities Report is coming up and I have prepared
> the skeleton for GHC's contribution [1]. If you have a project cooking
> or have recently had a patch land do have a look to make sure it's
> recognized in the submission.
>
> Cheers,
>
> - Ben
>
>
> https://ghc.haskell.org/trac/ghc/wiki/Status/Oct18
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

CI status

[Ben Gamari]

Hi everyone,

As you likely have noticed, GHC's CI is a bit of a mess since the
Hadrian merge, especially for differentials based on commits prior to
the merge. The problem is a tiresome limitation of Harbormaster's CI
strategy [1].

I have taken this opportunity to finally move testing of Differentials
to CircleCI. Unfortunately this has taken longer than anticipated due to
more tiresome Phabricator issues. At the moment the only guidance I can
offer for those in need of CI is to sit tight; the problem is being
worked on.

Cheers,

- Ben



[1] Namely, Harbormaster attempts to reuse working trees to perform
builds. However, it fails to remove stale submodules, meaning any
attempt to checkout a post-Hadrian-merge commit fails.


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Suppressing False Incomplete Pattern Matching Warnings for Polymorphic Pattern Synonyms

[Sylvain Henry]

I've just found this related ticket: 
https://ghc.haskell.org/trac/ghc/ticket/14422



On 10/26/18 7:04 PM, Richard Eisenberg wrote:
Aha. So you're viewing complete sets as a type-directed property, 
where we can take a type and look up what complete sets of patterns of 
that type might be.


Then, when checking a pattern-match for completeness, we use the 
inferred type of the pattern, access its complete sets, and if these 
match up. (Perhaps an implementation may optimize this process.)


What I like about this approach is that it works well with GADTs, 
where, e.g., VNil is a complete set for type Vec a Zero but not for 
Vec a n.


I take back my claim of "No types!" then, as this does sound like it 
has the right properties.


For now, I don't want to get bogged down by syntax -- let's figure out 
how the idea should work first, and then we can worry about syntax.


Here's a stab at a formalization of this idea, written in metatheory, 
not Haskell:


Let C : Type -> Set of set of patterns. C will be the lookup function 
for complete sets. Suppose we have a pattern match, at type tau, 
matching against patterns Ps. Let S = C(tau). S is then a set of sets 
of patterns. The question is this: Is there a set s in S such that Ps 
is a superset of s? If yes, then the match is complete.


What do we think of this design? Of course, the challenge is in 
building C, but we'll tackle that next.


Richard

On Oct 26, 2018, at 5:20 AM, Sylvain Henry > wrote:


Sorry I wasn't clear. I'm not an expert on the topic but it seems to 
me that there are two orthogonal concerns:


1) How does the checker retrieve COMPLETE sets.

Currently it seems to "attach" them to data type constructors (e.g. 
Maybe). If instead it retrieved them by matching types (e.g. "Maybe 
a", "a") we could write:


{-# COMPLETE LL #-}

From an implementation point of view, it seems to me that finding 
complete sets would become similar to finding (flexible, overlapping) 
class instances. Pseudo-code:


class Complete a where
   conlikes :: [ConLike]
instance Complete (Maybe a) where
   conlikes = [Nothing @a, Just @a]
instance Complete (Maybe a) where
   conlikes = [N @a, J @a]
instance Complete a where
   conlikes = [LL @a]
...


2) COMPLETE set depending on the matched type.

It is a thread hijack from me but while we are thinking about 
refactoring COMPLETE pragmas to support polymorphism, maybe we could 
support this too. The idea is to build a different set of conlikes 
depending on the matched type. Pseudo-code:


instance Complete (Variant cs) where
   conlikes = [V @c | c <- cs] -- cs is a type list

(I don't really care about the pragma syntax)

Sorry for the thread hijack!
Regards,
Sylvain


On 10/26/18 5:59 AM, Richard Eisenberg wrote:
I'm afraid I don't understand what your new syntax means. And, while 
I know it doesn't work today, what's wrong (in theory) with


{-# COMPLETE LL #-}

No types! (That's a rare thing for me to extol...)

I feel I must be missing something here.

Thanks,
Richard

On Oct 25, 2018, at 12:24 PM, Sylvain Henry > wrote:


> In the case where all the patterns are polymorphic, a user must
> provide a type signature but we accept the definition regardless of
> the type signature they provide. 


Currently we can specify the type *constructor* in a COMPLETE pragma:

pattern J :: a -> Maybe apattern J a = Just apattern N :: Maybe 
apattern N = Nothing{-# COMPLETE N, J :: Maybe #-}



Instead if we could specify the type with its free vars, we could 
refer to them in conlike signatures:


{-# COMPLETE N, [J:: a -> Maybe a ] :: Maybe a #-}

The COMPLETE pragma for LL could be:

{-# COMPLETE [LL :: HasSrcSpan a => SrcSpan -> SrcSpanLess a -> a ] 
:: a #-}


I'm borrowing the list comprehension syntax on purpose because it 
would allow to define a set of conlikes from a type-list (see my 
request [1]):


{-# COMPLETE [V :: (c :< cs) => c -> Variant cs | c <- cs ] :: 
Variant cs #-}


>To make things more formal, when the pattern-match checker
> requests a set of constructors for some data type constructor T, the
> checker returns:
>
>* The original set of data constructors for T
>* Any COMPLETE sets of type T
>
> Note the use of the phrase *type constructor*. The return type of all
> constructor-like things in a COMPLETE set must all be headed by the
> same type constructor T. Since `LL`'s return type is simply a type
> variable `a`, this simply doesn't work with the design of COMPLETE
> sets.

Could we use a mechanism similar to instance resolution (with 
FlexibleInstances) for the checker to return matching COMPLETE sets 
instead?


--Sylvain


[1]https://mail.haskell.org/pipermail/ghc-devs/2018-July/016053.html
___
ghc-devs mailing list
ghc-devs@haskell.org 
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs




___
ghc-devs maili