Re: GHC 9.6.1 rejects previously working code

[Harendra Kumar]

Thanks Tom and Rodrigo.

That clarifies the problem. We will need to think which solution makes
better sense.

On Wed, 12 Apr 2023 at 15:01, Rodrigo Mesquita 
wrote:

> Indeed, this is included in the GHC 9.6.x Migration Guide
> <https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.6#transformers-06-new-monadtrans-quantified-context>
> .
>
> Unfortunately, I’m also not sure there is a solution for this particular
> where (T m) is only a Monad if m instances MonadIO.
> As Tom explained, under transformers 0.6 `T` no longer is a monad
> transformer.
>
> A few workarounds I can think of:
>
> - No longer instance `MonadTrans T`, and use a instance `MonadIO m =>
> MonadIO (T m)` instead.
>   Rationale: if you always require `m` to be `MonadIO`, perhaps the
> ability to always lift an `m` to `T m` with `liftIO` is sufficient.
>
> - Add the `MonadIO` instance to the `m` field of `T`, GADT style, `data T
> m a where T :: MonadIO m => m -> T m a`
>   Rational: You would no longer need `MonadIO` in the `Monad` instance,
> which will make it possible to instance `MonadTrans`.
>
> - Redefine your own `lift` regardless of `MonadTrans`
>
> Good luck!
> Rodrigo
>
> On 12 Apr 2023, at 10:10, Tom Ellis <
> tom-lists-haskell-cafe-2...@jaguarpaw.co.uk> wrote:
>
> On Wed, Apr 12, 2023 at 02:32:43PM +0530, Harendra Kumar wrote:
>
> instance MonadIO m => Monad (T m) where
>return = pure
>(>>=) = undefined
>
> instance MonadTrans T where
>lift = undefined
>
>
> I guess it's nothing to do with 9.6 per se, but rather the difference
> between
>
> *
> https://hackage.haskell.org/package/transformers-0.5.6.2/docs/Control-Monad-Trans-Class.html#t:MonadTrans
>
> *
> https://hackage.haskell.org/package/transformers-0.6.1.0/docs/Control-Monad-Trans-Class.html#t:MonadTrans
>
> I'm not sure I can see any solution for this.  A monad transformer `T`
> must give rise to a monad `T m` regardless of what `m` is.  If `T m`
> is only a monad when `MonadIO m` then `T` can't be a monad transformer
> (under transformers 0.6).
>
> Tom
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC 9.6.1 rejects previously working code

[Harendra Kumar]

The following code compiles with older compilers but does not compile with
GHC 9.6.1:

{-# LANGUAGE KindSignatures #-}
module A () where

import Control.Monad.IO.Class
import Control.Monad.Trans.Class

data T (m :: * -> *) a = T

instance Functor (T m) where
fmap f T = undefined

instance Applicative (T m) where
pure = undefined
(<*>) = undefined

instance MonadIO m => Monad (T m) where
return = pure
(>>=) = undefined

instance MonadTrans T where
lift = undefined

It fails with the following error:

xx.hs:20:10: error: [GHC-3]
• Could not deduce ‘MonadIO m’
arising from the head of a quantified constraint
arising from the superclasses of an instance declaration
  from the context: Monad m
bound by a quantified context at xx.hs:20:10-21
  Possible fix:
add (MonadIO m) to the context of a quantified context
• In the instance declaration for ‘MonadTrans T’
   |
20 | instance MonadTrans T where
   |  

What is the correct resolution for this?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance of small allocations via prim ops

[Harendra Kumar]

On Fri, 7 Apr 2023 at 02:18, Carter Schonwald 
wrote:

> That sounds like a worthy experiment!
>
> I  guess that would look like having an inline macro’d up path that checks
> if it can get the job done that falls back to the general code?
>
> Last I checked, the overhead for this sort of c call was on the order of
> 10nanoseconds or less which seems like it’d be very unlikely to be a
> bottleneck, but do you have any natural or artificial benchmark programs
> that would show case this?
>

I converted my example code into a loop and ran it a million times with a 1
byte array size (would be 8 bytes after alignment). So roughly 3 words
would be allocated per array, including the header and length. It took 5 ms
using the statically known size optimization which inlines the alloc
completely, and 10 ms using an unknown size (from program arg) which makes
a call to newByteArray# . That turns out to be of the order of 5ns more per
allocation. It does not sound like a big deal.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance of small allocations via prim ops

[Harendra Kumar]

I am confused by this flag. This flag allows us to allocate statically
known arrays sizes of <= n to be allocated from the current nursery block.
But looking at the code in allocateMightFail, as I interpret it, any size
array up to LARGE_OBJECT_THRESHOLD is anyway allocated from the current
nursery block. So why have this option? Why not fix this to
LARGE_OBJECT_THRESHOLD? Maybe I am missing something.

-harendra

On Fri, 7 Apr 2023 at 15:45, Harendra Kumar 
wrote:

>
>
> On Fri, 7 Apr 2023 at 12:57, Simon Peyton Jones <
> simon.peytonjo...@gmail.com> wrote:
>
>> > We are emitting a more efficient code when the size of the array is
>> smaller. And the threshold is governed by a compiler flag:
>>
>> It would be good if this was documented.  Perhaps in the Haddock for
>> `newByteArray#`?  Or where?
>>
>
> The flag is documented in the GHC user guide but the behavior would be
> better discoverable if `newByteArray#` mentions it.
>
> -harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance of small allocations via prim ops

[Harendra Kumar]

On Fri, 7 Apr 2023 at 12:57, Simon Peyton Jones 
wrote:

> > We are emitting a more efficient code when the size of the array is
> smaller. And the threshold is governed by a compiler flag:
>
> It would be good if this was documented.  Perhaps in the Haddock for
> `newByteArray#`?  Or where?
>

The flag is documented in the GHC user guide but the behavior would be
better discoverable if `newByteArray#` mentions it.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance of small allocations via prim ops

[Harendra Kumar]

Little bit of grepping in the code gave me this:

emitPrimOp cfg primop =
  let max_inl_alloc_size = fromIntegral (stgToCmmMaxInlAllocSize cfg)
  in case primop of
  NewByteArrayOp_Char -> \case
[(CmmLit (CmmInt n w))]
  | asUnsigned w n <= max_inl_alloc_size --
<--- see this line
  -> opIntoRegs  $ \ [res] -> doNewByteArrayOp res (fromInteger n)
_ -> PrimopCmmEmit_External

We are emitting a more efficient code when the size of the array is
smaller. And the threshold is governed by a compiler flag:

  , make_ord_flag defGhcFlag "fmax-inline-alloc-size"
  (intSuffix (\n d -> d { maxInlineAllocSize = n }))

This means allocation of smaller arrays is extremely efficient and we can
control it using `-fmax-inline-alloc-size`, the default is 128. That's a
new thing I learnt today.

Given this new finding, my original question now applies only to the case
when the array size is bigger than this configurable threshold, which is a
little less motivating. And Ben says that the call is not expensive, so we
can leave it there.

-harendra

On Fri, 7 Apr 2023 at 11:08, Harendra Kumar 
wrote:

> Ah, some other optimization seems to be kicking in here. When I increase
> the size of the array to > 128 then I see a call to stg_newByteArray# being
> emitted:
>
>  {offset
>c1kb: // global
>if ((Sp + -8) < SpLim) (likely: False) goto c1kc; else goto
> c1kd;
>c1kc: // global
>R1 = Main.main1_closure;
>call (stg_gc_fun)(R1) args: 8, res: 0, upd: 8;
>c1kd: // global
>I64[Sp - 8] = c1k9;
>R1 = 129;
>Sp = Sp - 8;
>call stg_newByteArray#(R1) returns to c1k9, args: 8, res: 8,
> upd: 8;
>
> -harendra
>
> On Fri, 7 Apr 2023 at 10:49, Harendra Kumar 
> wrote:
>
>> Thanks Ben and Carter.
>>
>> I compiled the following to Cmm:
>>
>> {-# LANGUAGE MagicHash #-}
>> {-# LANGUAGE UnboxedTuples #-}
>>
>> import GHC.IO
>> import GHC.Exts
>>
>> data M = M (MutableByteArray# RealWorld)
>>
>> main = do
>>  _ <- IO (\s -> case newByteArray# 1# s of (# s1, arr #) -> (# s1, M
>> arr #))
>>  return ()
>>
>> It produced the following Cmm:
>>
>>  {offset
>>c1k3: // global
>>Hp = Hp + 24;
>>if (Hp > HpLim) (likely: False) goto c1k7; else goto c1k6;
>>c1k7: // global
>>HpAlloc = 24;
>>R1 = Main.main1_closure;
>>call (stg_gc_fun)(R1) args: 8, res: 0, upd: 8;
>>c1k6: // global
>>I64[Hp - 16] = stg_ARR_WORDS_info;
>>I64[Hp - 8] = 1;
>>R1 = GHC.Tuple.()_closure+1;
>>call (P64[Sp])(R1) args: 8, res: 0, upd: 8;
>>  }
>>
>> It seems to be as good as it gets. There is absolutely no scope for
>> improvement in this.
>>
>> -harendra
>>
>> On Fri, 7 Apr 2023 at 03:32, Ben Gamari  wrote:
>>
>>> Harendra Kumar  writes:
>>>
>>> > I was looking at the RTS code for allocating small objects via prim ops
>>> > e.g. newByteArray# . The code looks like:
>>> >
>>> > stg_newByteArrayzh ( W_ n )
>>> > {
>>> > MAYBE_GC_N(stg_newByteArrayzh, n);
>>> >
>>> > payload_words = ROUNDUP_BYTES_TO_WDS(n);
>>> > words = BYTES_TO_WDS(SIZEOF_StgArrBytes) + payload_words;
>>> > ("ptr" p) = ccall allocateMightFail(MyCapability() "ptr", words);
>>> >
>>> > We are making a foreign call here (ccall). I am wondering how much
>>> overhead
>>> > a ccall adds? I guess it may have to save and restore registers. Would
>>> it
>>> > be better to do the fast path case of allocating small objects from the
>>> > nursery using cmm code like in stg_gc_noregs?
>>> >
>>> GHC's operational model is designed in such a way that foreign calls are
>>> fairly cheap (e.g. we don't need to switch stacks, which can be quite
>>> costly). Judging by the assembler produced for newByteArray# in one
>>> random x86-64 tree that I have lying around, it's only a couple of
>>> data-movement instructions, an %eax clear, and a stack pop:
>>>
>>>   36:   48 89 cemov%rcx,%rsi
>>>   39:   48 89 c7mov%rax,%rdi
>>>   3c:   31 c0   xor%eax,%eax
>>>   3e:   e8 00 00 00 00  call   43
>>> 
>>>   43:   48 83 c4 08 add$0x8,%rsp
>>>
>>> The data movement operations in particular are quite cheap on most
>>> microarchitectures where GHC would run due to register renaming. I doubt
>>> that this overhead would be noticable in anything but a synthetic
>>> benchmark. However, it never hurts to measure.
>>>
>>> Cheers,
>>>
>>> - Ben
>>>
>>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance of small allocations via prim ops

[Harendra Kumar]

Ah, some other optimization seems to be kicking in here. When I increase
the size of the array to > 128 then I see a call to stg_newByteArray# being
emitted:

 {offset
   c1kb: // global
   if ((Sp + -8) < SpLim) (likely: False) goto c1kc; else goto c1kd;
   c1kc: // global
   R1 = Main.main1_closure;
   call (stg_gc_fun)(R1) args: 8, res: 0, upd: 8;
   c1kd: // global
   I64[Sp - 8] = c1k9;
   R1 = 129;
   Sp = Sp - 8;
   call stg_newByteArray#(R1) returns to c1k9, args: 8, res: 8,
upd: 8;

-harendra

On Fri, 7 Apr 2023 at 10:49, Harendra Kumar 
wrote:

> Thanks Ben and Carter.
>
> I compiled the following to Cmm:
>
> {-# LANGUAGE MagicHash #-}
> {-# LANGUAGE UnboxedTuples #-}
>
> import GHC.IO
> import GHC.Exts
>
> data M = M (MutableByteArray# RealWorld)
>
> main = do
>  _ <- IO (\s -> case newByteArray# 1# s of (# s1, arr #) -> (# s1, M
> arr #))
>  return ()
>
> It produced the following Cmm:
>
>  {offset
>c1k3: // global
>Hp = Hp + 24;
>if (Hp > HpLim) (likely: False) goto c1k7; else goto c1k6;
>c1k7: // global
>HpAlloc = 24;
>R1 = Main.main1_closure;
>call (stg_gc_fun)(R1) args: 8, res: 0, upd: 8;
>c1k6: // global
>I64[Hp - 16] = stg_ARR_WORDS_info;
>I64[Hp - 8] = 1;
>R1 = GHC.Tuple.()_closure+1;
>call (P64[Sp])(R1) args: 8, res: 0, upd: 8;
>  }
>
> It seems to be as good as it gets. There is absolutely no scope for
> improvement in this.
>
> -harendra
>
> On Fri, 7 Apr 2023 at 03:32, Ben Gamari  wrote:
>
>> Harendra Kumar  writes:
>>
>> > I was looking at the RTS code for allocating small objects via prim ops
>> > e.g. newByteArray# . The code looks like:
>> >
>> > stg_newByteArrayzh ( W_ n )
>> > {
>> > MAYBE_GC_N(stg_newByteArrayzh, n);
>> >
>> > payload_words = ROUNDUP_BYTES_TO_WDS(n);
>> > words = BYTES_TO_WDS(SIZEOF_StgArrBytes) + payload_words;
>> > ("ptr" p) = ccall allocateMightFail(MyCapability() "ptr", words);
>> >
>> > We are making a foreign call here (ccall). I am wondering how much
>> overhead
>> > a ccall adds? I guess it may have to save and restore registers. Would
>> it
>> > be better to do the fast path case of allocating small objects from the
>> > nursery using cmm code like in stg_gc_noregs?
>> >
>> GHC's operational model is designed in such a way that foreign calls are
>> fairly cheap (e.g. we don't need to switch stacks, which can be quite
>> costly). Judging by the assembler produced for newByteArray# in one
>> random x86-64 tree that I have lying around, it's only a couple of
>> data-movement instructions, an %eax clear, and a stack pop:
>>
>>   36:   48 89 cemov%rcx,%rsi
>>   39:   48 89 c7mov%rax,%rdi
>>   3c:   31 c0   xor%eax,%eax
>>   3e:   e8 00 00 00 00  call   43
>> 
>>   43:   48 83 c4 08 add$0x8,%rsp
>>
>> The data movement operations in particular are quite cheap on most
>> microarchitectures where GHC would run due to register renaming. I doubt
>> that this overhead would be noticable in anything but a synthetic
>> benchmark. However, it never hurts to measure.
>>
>> Cheers,
>>
>> - Ben
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance of small allocations via prim ops

[Harendra Kumar]

Thanks Ben and Carter.

I compiled the following to Cmm:

{-# LANGUAGE MagicHash #-}
{-# LANGUAGE UnboxedTuples #-}

import GHC.IO
import GHC.Exts

data M = M (MutableByteArray# RealWorld)

main = do
 _ <- IO (\s -> case newByteArray# 1# s of (# s1, arr #) -> (# s1, M
arr #))
 return ()

It produced the following Cmm:

 {offset
   c1k3: // global
   Hp = Hp + 24;
   if (Hp > HpLim) (likely: False) goto c1k7; else goto c1k6;
   c1k7: // global
   HpAlloc = 24;
   R1 = Main.main1_closure;
   call (stg_gc_fun)(R1) args: 8, res: 0, upd: 8;
   c1k6: // global
   I64[Hp - 16] = stg_ARR_WORDS_info;
   I64[Hp - 8] = 1;
   R1 = GHC.Tuple.()_closure+1;
   call (P64[Sp])(R1) args: 8, res: 0, upd: 8;
 }

It seems to be as good as it gets. There is absolutely no scope for
improvement in this.

-harendra

On Fri, 7 Apr 2023 at 03:32, Ben Gamari  wrote:

> Harendra Kumar  writes:
>
> > I was looking at the RTS code for allocating small objects via prim ops
> > e.g. newByteArray# . The code looks like:
> >
> > stg_newByteArrayzh ( W_ n )
> > {
> > MAYBE_GC_N(stg_newByteArrayzh, n);
> >
> > payload_words = ROUNDUP_BYTES_TO_WDS(n);
> > words = BYTES_TO_WDS(SIZEOF_StgArrBytes) + payload_words;
> > ("ptr" p) = ccall allocateMightFail(MyCapability() "ptr", words);
> >
> > We are making a foreign call here (ccall). I am wondering how much
> overhead
> > a ccall adds? I guess it may have to save and restore registers. Would it
> > be better to do the fast path case of allocating small objects from the
> > nursery using cmm code like in stg_gc_noregs?
> >
> GHC's operational model is designed in such a way that foreign calls are
> fairly cheap (e.g. we don't need to switch stacks, which can be quite
> costly). Judging by the assembler produced for newByteArray# in one
> random x86-64 tree that I have lying around, it's only a couple of
> data-movement instructions, an %eax clear, and a stack pop:
>
>   36:   48 89 cemov%rcx,%rsi
>   39:   48 89 c7mov%rax,%rdi
>   3c:   31 c0   xor%eax,%eax
>   3e:   e8 00 00 00 00  call   43 
>   43:   48 83 c4 08 add$0x8,%rsp
>
> The data movement operations in particular are quite cheap on most
> microarchitectures where GHC would run due to register renaming. I doubt
> that this overhead would be noticable in anything but a synthetic
> benchmark. However, it never hurts to measure.
>
> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Performance of small allocations via prim ops

[Harendra Kumar]

I was looking at the RTS code for allocating small objects via prim ops
e.g. newByteArray# . The code looks like:

stg_newByteArrayzh ( W_ n )
{
MAYBE_GC_N(stg_newByteArrayzh, n);

payload_words = ROUNDUP_BYTES_TO_WDS(n);
words = BYTES_TO_WDS(SIZEOF_StgArrBytes) + payload_words;
("ptr" p) = ccall allocateMightFail(MyCapability() "ptr", words);

We are making a foreign call here (ccall). I am wondering how much overhead
a ccall adds? I guess it may have to save and restore registers. Would it
be better to do the fast path case of allocating small objects from the
nursery using cmm code like in stg_gc_noregs?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC API documentation

[Harendra Kumar]

Thanks Ben.

The link is broken in GHC 9.4.4 as well. But works in 9.2.x .

On Tue, 14 Mar 2023 at 23:19, Ben Gamari  wrote:

> Harendra Kumar  writes:
>
> > Hi,
> >
> >
> > There used to be a GHC API documentation along with other documentation
> in
> > the GHC release. But I cannot find it any more. For example, in GHC 9.6.1
> > this link -
> > https://downloads.haskell.org/ghc/9.6.1/docs/libraries/ghc/index.html is
> > broken.
> >
> > What happened to this? Is this now available at some alternate location?
> >
> The URL is now
> https://downloads.haskell.org/ghc/9.6.1/docs/libraries/ghc-9.6.1/index.html
> .
>
> However, it is rather concerning that the link on
> https://downloads.haskell.org/ghc/9.6.1/docs/ is broken.
> I've opened #23121 to track this.
>
> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC API documentation

[Harendra Kumar]

Hi,


There used to be a GHC API documentation along with other documentation in
the GHC release. But I cannot find it any more. For example, in GHC 9.6.1
this link -
https://downloads.haskell.org/ghc/9.6.1/docs/libraries/ghc/index.html is
broken.

What happened to this? Is this now available at some alternate location?


-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Instructions to build GHC

[Harendra Kumar]

Hi,

I used to follow the instructions in INSTALL.md and README.md in the
root of the repo, and they always worked until recently. I am aware
that the make based system has been replaced by Hadrian. But it seems
these files are not updated with the new build instructions.

1. Either these files should be removed from the source or they should
be updated.
2. Where are the new build instructions? Is it hadrian/README.md? If
so can we put a pointer to that in INSTALL.md?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Wiki page for WebAssembly support in GHC

[Harendra Kumar]

This is exciting news. Thank you Cheng and Norman, and Tweag!

With both GHCJS and WASM officially supported in GHC, Haskell will now
have first class support for JS. I am looking forward to GHC 9.6.

-harendra

On Thu, 3 Mar 2022 at 19:42, Cheng Shao  wrote:
>
> Hi all,
>
> We're planning to add WebAssembly support to GHC. This work will be
> delivered by me and Norman Ramsey, the former Asterius team at Tweag.
>
> In the past few months, we did a strategic shift and focused on
> growing GHC towards Asterius, leveraging existing RTS as much as
> possible. We've made decent progress since then, and we expect to
> target 9.6 release much like GHCJS.
>
> We've created the
> https://gitlab.haskell.org/ghc/ghc/-/wikis/WebAssembly-backend wiki
> page, as a central point for communication. You're welcome to take a
> look and ask questions (either in this thread or editing that page in
> place), we'll answer the questions in the FAQ section.
>
> Cheers,
> Cheng
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: downloading ghc head version

[Harendra Kumar]

Yes, it is working with deb10. All good. Not sure why
haskell/actions/setup is still using the deb9 URL.

Thanks for your help.

On Tue, 1 Feb 2022 at 17:29, Matthew Pickering
 wrote:
>
> Right, it appears the problem was we stopped producing the deb9
> bindists due to deb9 being EOL.
>
> https://gitlab.haskell.org/ghc/ghc/-/commit/871ce2a300ed35639a39a86f4c85fbcb605c5d7d
>
> Is your problem sorted now?
>
> Matt
>
> On Tue, Feb 1, 2022 at 12:09 AM Harendra Kumar  
> wrote:
> >
> > I replaced deb9 with deb10 in the URL that ghcup install was using
> > earlier in the CI and it seems to work. The following command
> > installed ghc successfully:
> >
> > $ ghcup install ghc -u
> > https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-deb10-linux-integer-simple.tar.xz?job=validate-x86_64-linux-deb10-integer-simple
> > head
> > $ ghc-head --version
> > The Glorious Glasgow Haskell Compilation System, version 9.3.20220130
> >
> > Maybe the deb9 build has been retired now? It worked once in our CI a
> > few days and then never worked.
> >
> > -harendra
> >
> > On Tue, 1 Feb 2022 at 05:24, Harendra Kumar  
> > wrote:
> > >
> > > Ah, nice. That looks like a useful repo. Thanks!
> > >
> > > I tried the nix expression:
> > >
> > > nix run -f 
> > > https://github.com/mpickering/ghc-artefact-nix/archive/master.tar.gz
> > > ghcHEAD
> > >
> > > on my debian linux, but it is trying to download the fedora tar from
> > > https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-fedora33-linux.tar.xz?job=validate-x86_64-linux-fedora33
> > > . ghc-head-from.sh also downloads from the same link. Is it supposed
> > > to work on debian as well? The nix expression as well as
> > > ghc-head-from.sh finally failed with the followinf error on my
> > > machine:
> > >
> > > utils/ghc-cabal/dist-install/build/tmp/ghc-cabal: error while loading
> > > shared libraries: libstdc++.so.6: cannot open shared object file: No
> > > such file or directory
> > >
> > > I looked at 
> > > https://github.com/mpickering/ghc-artefact-nix/blob/master/gitlab-artifact.nix,
> > > but it does not seem to have a debian x86_64 config.
> > >
> > > -harendra
> > >
> > > On Tue, 1 Feb 2022 at 05:00, Matthew Pickering
> > >  wrote:
> > > >
> > > > I will look tomorrow but see where ghc-head-from gets the bindist from
> > > >
> > > > https://github.com/mpickering/ghc-artefact-nix/blob/master/ghc-head-from.sh
> > > >
> > > > On Mon, Jan 31, 2022 at 10:43 PM Harendra Kumar
> > > >  wrote:
> > > > >
> > > > > It seems the latest artifacts download link
> > > > > (https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/...) at GHC
> > > > > gitlab is not working.
> > > > >
> > > > > If this is not the right place to ask this, can someone point me to
> > > > > the right place?
> > > > >
> > > > > -harendra
> > > > >
> > > > > On Wed, 26 Jan 2022 at 18:38, Harendra Kumar 
> > > > >  wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am trying to set up a CI for ghc head version. I am not sure what 
> > > > > > is
> > > > > > the official supported method to install a nightly/head version of
> > > > > > GHC. haskell/actions/setup on github seems to support it via ghcup.
> > > > > > But it almost always fails, I saw it succeeding once till now. It
> > > > > > tries to download it from the following URL, but fails with 404 not
> > > > > > found:
> > > > > >
> > > > > > downloading: 
> > > > > > https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-deb9-linux-integer-simple.tar.xz?job=validate-x86_64-linux-deb9-integer-simple
> > > > > >
> > > > > > I tried this link in the browser and I get the same error. Is that 
> > > > > > the
> > > > > > right way to install it? If not, can someone suggest a reliable way 
> > > > > > to
> > > > > > get the head version, other than building it myself?
> > > > > >
> > > > > > Thanks,
> > > > > > Harendra
> > > > > ___
> > > > > ghc-devs mailing list
> > > > > ghc-devs@haskell.org
> > > > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: downloading ghc head version

[Harendra Kumar]

It seems the latest artifacts download link
(https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/...) at GHC
gitlab is not working.

If this is not the right place to ask this, can someone point me to
the right place?

-harendra

On Wed, 26 Jan 2022 at 18:38, Harendra Kumar  wrote:
>
> Hi,
>
> I am trying to set up a CI for ghc head version. I am not sure what is
> the official supported method to install a nightly/head version of
> GHC. haskell/actions/setup on github seems to support it via ghcup.
> But it almost always fails, I saw it succeeding once till now. It
> tries to download it from the following URL, but fails with 404 not
> found:
>
> downloading: 
> https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-deb9-linux-integer-simple.tar.xz?job=validate-x86_64-linux-deb9-integer-simple
>
> I tried this link in the browser and I get the same error. Is that the
> right way to install it? If not, can someone suggest a reliable way to
> get the head version, other than building it myself?
>
> Thanks,
> Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

downloading ghc head version

[Harendra Kumar]

Hi,

I am trying to set up a CI for ghc head version. I am not sure what is
the official supported method to install a nightly/head version of
GHC. haskell/actions/setup on github seems to support it via ghcup.
But it almost always fails, I saw it succeeding once till now. It
tries to download it from the following URL, but fails with 404 not
found:

downloading: 
https://gitlab.haskell.org/ghc/ghc/-/jobs/artifacts/master/raw/ghc-x86_64-deb9-linux-integer-simple.tar.xz?job=validate-x86_64-linux-deb9-integer-simple

I tried this link in the browser and I get the same error. Is that the
right way to install it? If not, can someone suggest a reliable way to
get the head version, other than building it myself?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Measuring thread cpu time (using bound threads)

[Harendra Kumar]

Hi GHC devs,

I want to measure the CPU time spent in a particular Haskell thread across
thread yield points. I am assuming Haskell threads do not keep track of per
thread cpu time. Therefore, to measure it I have to use the thread cpu
clock (CLOCK_THREAD_CPUTIME_ID) provided by the OS (Linux). But Haskell
threads can keep jumping across multiple OS threads so the OS thread's CPU
time cannot be used reliably.

To solve that issue I thought I could use bound threads. From the
documentation it sounded like bound threads are exclusively bound to a
particular OS thread so we can get thread cpu time from the OS and use it.

However, if I print the thread id for a bound thread using "gettid" (OS
thread id on Linux) or using "pthread_self" (pthread id), both these thread
ids are changing for the same Haskell thread which is a bound thread. Is
that expected, or am I doing something wrong?

My questions:

* Are bound threads really bound to a particular OS thread or they can use
multiple OS threads for executing Haskell code, and only use a fixed OS
thread for FFI code?
* Is there a way to make this work? Can we obtain the thread cpu time
reliably across thread yield points? Is there a way to really bind Haskell
threads to OS threads?
* Is there a better or alternative way to get the thread CPU time?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Compiling "primitive" with ghc head

[Harendra Kumar]

Forgot to add subject in the previous email.

On Sat, 27 Nov 2021 at 01:01, Harendra Kumar 
wrote:

> Hi GHC devs,
>
> While compiling the primitive package using ghc head I ran into the
> following error:
>
> Data/Primitive/Types.hs:265:870: error:
> • Couldn't match type ‘Word64#’ with ‘Word#’
>   Expected: Word64_#
> Actual: Word64#
> • In the fourth argument of ‘setWord64Array#’, namely ‘x#’
>   In the first argument of ‘internal’, namely
> ‘(setWord64Array# arr# i n x#)’
>   In the first argument of ‘unsafeCoerce#’, namely
> ‘(internal (setWord64Array# arr# i n x#))’
> |
> 265 | derivePrim(Word64, W64#, sIZEOF_WORD64, aLIGNMENT_WORD64,
> |
>
>
>
> Any idea what this is and how it can be fixed?
>
> -harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[no subject]

[Harendra Kumar]

Hi GHC devs,

While compiling the primitive package using ghc head I ran into the
following error:

Data/Primitive/Types.hs:265:870: error:
• Couldn't match type ‘Word64#’ with ‘Word#’
  Expected: Word64_#
Actual: Word64#
• In the fourth argument of ‘setWord64Array#’, namely ‘x#’
  In the first argument of ‘internal’, namely
‘(setWord64Array# arr# i n x#)’
  In the first argument of ‘unsafeCoerce#’, namely
‘(internal (setWord64Array# arr# i n x#))’
|
265 | derivePrim(Word64, W64#, sIZEOF_WORD64, aLIGNMENT_WORD64,
|



Any idea what this is and how it can be fixed?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Question about specialization

[Harendra Kumar]

I am in the middle of a refactor. I will file a ticket once I am done.

On Mon, 6 Sept 2021 at 20:17, Simon Peyton Jones 
wrote:

> Harendra
>
>
>
> That comes as a surprise to me.  Could you possibly make a repo case, and
> say what version of the compiler does, and does not, specialise the
> function?
>
>
>
> File it as a ticket … to me it looks like a bug.
>
>
>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* ghc-devs  *On Behalf Of *Harendra
> Kumar
> *Sent:* 06 September 2021 14:11
> *To:* ghc-devs@haskell.org
> *Subject:* Question about specialization
>
>
>
> Hi GHC devs,
>
>
>
> I have a simple program using the streamly library, as follows, the whole
> code is in the same module:
>
>
>
> {-# INLINE iterateState #-}
> {-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-}
> iterateState :: MonadState Int m => Int -> SerialT m Int
> iterateState n = do
> x <- get
> if x > n
> then do
> put (x - 1)
> iterateState n
> else return x
>
>
>
> main :: IO ()
> main = do
> State.evalStateT (S.drain (iterateState 0)) 10
>
>
>
> Earlier the SPECIALIZE pragma was not required on iterateState, but after
> some refactoring in the library (the monad bind of SerialT changed a bit),
> this program now requires a SPECIALIZE on iterateState to trigger
> specialization, just INLINE also does not help.
>
>
>
> My question is whether this may be expected in some conditions or is this
> something which can be considered a bug in the compiler? I am also curious
> what specifically could have made the compiler not specialize this anymore,
> is it the size of the function or some other threshold?
>
>
>
> Thanks,
>
> Harendra
>
>
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Question about specialization

[Harendra Kumar]

Hi GHC devs,

I have a simple program using the streamly library, as follows, the whole
code is in the same module:

{-# INLINE iterateState #-}
{-# SPECIALIZE iterateState :: Int -> SerialT (StateT Int IO) Int #-}
iterateState :: MonadState Int m => Int -> SerialT m Int
iterateState n = do
x <- get
if x > n
then do
put (x - 1)
iterateState n
else return x

main :: IO ()
main = do
State.evalStateT (S.drain (iterateState 0)) 10

Earlier the SPECIALIZE pragma was not required on iterateState, but after
some refactoring in the library (the monad bind of SerialT changed a bit),
this program now requires a SPECIALIZE on iterateState to trigger
specialization, just INLINE also does not help.

My question is whether this may be expected in some conditions or is this
something which can be considered a bug in the compiler? I am also curious
what specifically could have made the compiler not specialize this anymore,
is it the size of the function or some other threshold?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Breaking changes to the base library

[Harendra Kumar]

Just an aside. I looked up the GHC user guide and found this:

-Wcompat-unqualified-imports

Warns on qualified imports of core library modules which are subject to
change in future GHC releases.
I got confused by the text. I guess it was meant to be "Warns on
unqualified imports".
-harendra

On Sun, 20 Jun 2021 at 20:39, Oleg Grenrus  wrote:

> What is the reason that -Wcompat flags are not enabled by -Wall? The
> unqualified Data.List import was warned by -Wcompat about since GHC-8.10,
> I'm surprised that it still surprise people.
>
> Should -Wcompat be implied by -Wall? Why not? I think that people who
> won't ever consider upgrading GHC to be a very very small minority, and
> they can suppress these warnings with `-Wall -Wno-compat`.
>
> - Oleg
>
> On 20.6.2021 17.57, Edward Kmett wrote:
>
> The breakage concern is why Data.List wound up in its limbo-like state of
> re-exporting the Foldable-polymorphic combinators since 7.10 -- while
> "weird", it was the only option that didn't have to choose between removing
> the names from Data.List exports entirely and breaking unqualified imports
> of Data.List.
>
> With the monomorphized combinators in place, Data.List should be
> considered a 'qualified' import like Data.Map. We definitely need to do
> more to communicate that this is changing and how users should adjust their
> code to suit. After all, by far the most common intended import from
> Data.List is the humble 'sort', which doesn't conflict.
>
> -Edward
>
> On Sun, Jun 20, 2021 at 4:45 AM Chris Smith  wrote:
>
>> Yikes, this is going to break nearly everything.  Definitely good to let
>> people know.
>>
>> On Sun, Jun 20, 2021 at 7:43 AM Ben Gamari  wrote:
>>
>>> Harendra Kumar  writes:
>>>
>>> > I see the following errors when compiling with ghc head version:
>>> >
>>> > $ ghc-stage2 --version
>>> > The Glorious Glasgow Haskell Compilation System, version 9.3.20210608
>>> >
>>> > $ cabal build --with-compiler ghc-stage2 --allow-newer
>>> >
>>> > Data/Colour/CIE.hs:80:12: error:
>>> > Ambiguous occurrence ‘sum’
>>> > It could refer to
>>> >either ‘Prelude.sum’,
>>> >   imported from ‘Prelude’ at Data/Colour/CIE.hs:25:8-22
>>> >   (and originally defined in ‘Data.Foldable’)
>>> >or ‘Data.List.sum’,
>>> >   imported from ‘Data.List’ at Data/Colour/CIE.hs:41:1-16
>>> >   (and originally defined in ‘GHC.List’)
>>> >|
>>> > 80 |total = sum $ map fst l
>>> >|^^^
>>> >
>>> > Can someone briefly describe this change and what's the recommended
>>> way of
>>> > fixing this? Just hide the Data.List definition? I do not see this
>>> > mentioned in the release notes of 9.2/9.4 here:
>>> > https://ghc.gitlab.haskell.org/ghc/doc/users_guide/9.2.1-notes.html
>>> >
>>> Indeed, this is due to the monomorphic Data.List proposal, which the
>>> CLC decided would accompany the addition of Data.List.singleton. The
>>> correct fix here is to either qualify the import of `Data.List` or add
>>> an explicit import list. I'll try to remember to add a note about this
>>> to the release notes and migration guide.
>>>
>>>
>>> Cheers,
>>>
>>> - Ben
>>>
>>> ___
>>> ghc-devs mailing list
>>> ghc-devs@haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
> ___
> ghc-devs mailing 
> listghc-devs@haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Breaking changes to the base library

[Harendra Kumar]

I see the following errors when compiling with ghc head version:

$ ghc-stage2 --version
The Glorious Glasgow Haskell Compilation System, version 9.3.20210608

$ cabal build --with-compiler ghc-stage2 --allow-newer

Data/Colour/CIE.hs:80:12: error:
Ambiguous occurrence ‘sum’
It could refer to
   either ‘Prelude.sum’,
  imported from ‘Prelude’ at Data/Colour/CIE.hs:25:8-22
  (and originally defined in ‘Data.Foldable’)
   or ‘Data.List.sum’,
  imported from ‘Data.List’ at Data/Colour/CIE.hs:41:1-16
  (and originally defined in ‘GHC.List’)
   |
80 |total = sum $ map fst l
   |^^^

Can someone briefly describe this change and what's the recommended way of
fixing this? Just hide the Data.List definition? I do not see this
mentioned in the release notes of 9.2/9.4 here:
https://ghc.gitlab.haskell.org/ghc/doc/users_guide/9.2.1-notes.html

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC head, profiling with -O2 does not work

[Harendra Kumar]

Hi,

I am trying to build a small program with profiling using a ghc built from
haskell gitlab master branch. When I use -O2 I run into:

$ ghc/inplace/bin/ghc-stage2 --make -O2 -prof -fprof-auto -rtsopts Main.hs
Linking Main ...
./StreamD.o(.text+0x1b9f2): error: undefined reference to
'StreamK_mkStreamFromStream_HPC_cc'
./MArray.o(.text+0xbe83): error: undefined reference to
'StreamK_mkStreamFromStream_HPC_cc'
Main.o(.text+0x6fdb): error: undefined reference to
'StreamK_mkStreamFromStream_HPC_cc'
collect2: error: ld returned 1 exit status
`cc' failed in phase `Linker'. (Exit code: 1)

It builds fine without the -O2 option. On ghc-8 it builds fine even with
-O2. Is that a bug in the head GHC or just something to do with how I built
it?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf regression with TypeFamilies

[Harendra Kumar]

I opened a GHC ticket (https://gitlab.haskell.org/ghc/ghc/-/issues/18414)
with full details.

-harendra

On Tue, 30 Jun 2020 at 18:25, Harendra Kumar 
wrote:

> Hi Richard,
>
> I am glad that you are interested in it. It is the runtime performance
> that degrades, and yes it is fully reproducible and publicly shareable, the
> library is open source so anyone can run these benchmarks. I had postponed
> the problem raising an issue to investigate it further here
> https://github.com/composewell/streamly/issues/567 . Since you are
> interested I will investigate it sooner. I have added some perf numbers in
> that issue.
>
> I was also surprised by it, thinking what can type families possibly do to
> degrade run time performance like that. I can try compiling one worst
> affected benchmark code and look at the core-2-core passes to see where it
> makes the difference.
>
> -harendra
>
> On Mon, 29 Jun 2020 at 19:45, Richard Eisenberg  wrote:
>
>> Hi Harendra,
>>
>> I saw your comment on a ghc proposal (
>> https://github.com/ghc-proposals/ghc-proposals/pull/343#issuecomment-650797297)
>> that said you experienced perf regressions with -XTypeFamilies enabled (but
>> no other changes). Are these reproducible in a way that can be shared
>> publicly? And are the regressions in compile times or run times?
>>
>> -XTypeFamilies enables -XMonoLocalBinds, which disables
>> let-generalization on some nested lets. It is thus just barely conceivable
>> that different Core is produced depending on this extension, and that there
>> may be a possibility of performance changes. But this would be unexpected,
>> and something worth investigating.
>>
>> In other words: if you can, do please post a bug -- simply enabling
>> -XTypeFamilies should not slow anything down!
>>
>> Thanks,
>> Richard
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: perf regression with TypeFamilies

[Harendra Kumar]

Hi Richard,

I am glad that you are interested in it. It is the runtime performance that
degrades, and yes it is fully reproducible and publicly shareable, the
library is open source so anyone can run these benchmarks. I had postponed
the problem raising an issue to investigate it further here
https://github.com/composewell/streamly/issues/567 . Since you are
interested I will investigate it sooner. I have added some perf numbers in
that issue.

I was also surprised by it, thinking what can type families possibly do to
degrade run time performance like that. I can try compiling one worst
affected benchmark code and look at the core-2-core passes to see where it
makes the difference.

-harendra

On Mon, 29 Jun 2020 at 19:45, Richard Eisenberg  wrote:

> Hi Harendra,
>
> I saw your comment on a ghc proposal (
> https://github.com/ghc-proposals/ghc-proposals/pull/343#issuecomment-650797297)
> that said you experienced perf regressions with -XTypeFamilies enabled (but
> no other changes). Are these reproducible in a way that can be shared
> publicly? And are the regressions in compile times or run times?
>
> -XTypeFamilies enables -XMonoLocalBinds, which disables let-generalization
> on some nested lets. It is thus just barely conceivable that different Core
> is produced depending on this extension, and that there may be a
> possibility of performance changes. But this would be unexpected, and
> something worth investigating.
>
> In other words: if you can, do please post a bug -- simply enabling
> -XTypeFamilies should not slow anything down!
>
> Thanks,
> Richard
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Fusing loops by specializing on functions with SpecConstr?

[Harendra Kumar]

On Wed, 1 Apr 2020 at 02:49, Alexis King  wrote:

>
> I’ve been trying to figure out if it would be possible to help the
> optimizer out by annotating the program with special combinators like the
> existing ones provided by GHC.Magic. However, I haven’t been able to come
> up with anything yet that seems like it would actually work.
>

You may want to take a look at https://github.com/composewell/fusion-plugin
which uses annotations to help GHC fuse, not specifically what you want but
might possibly be relevant to your work.
https://github.com/composewell/streamly relies heavily on case-of-case and
SpecConstr for stream fusion. There are several cases that GHC is unable to
fuse currently. We use a "Fuse" annotation to tell GHC that any function
involving this type must be inlined so that fusion can occur reliably. With
the help of fusion-plugin we have been able to fuse almost every known case
in streamly till now.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Random crashes with memory corruption symptoms

[Harendra Kumar]

Ok, I will file an issue. I just wanted to rule out any application level
issues first. Did you try it on the code I sent or did you use some other
test case? Is there a test-suite in GHC that stresses the threaded runtime?

-harendra

On Mon, 3 Feb 2020 at 14:09, Ömer Sinan Ağacan  wrote:

> In that case it'd be good to move the discussion to Gitlab. Could you file
> an
> issue?
>
> I was able to reproduce on GHC HEAD. With debug runtime I consistently get
> this
> assertion error:
>
> internal error: ASSERTION FAILED: file rts/Messages.c, line 95
>
> (GHC version 8.11.0.20200201 for x86_64_unknown_linux)
> Please report this as a GHC bug:
> https://www.haskell.org/ghc/reportabug
>
> In non-debug runtime it works fine maybe half of the time, in others I get
> a
> panic in the GC.
>
> Ömer
>
> Harendra Kumar , 3 Şub 2020 Pzt, 10:01
> tarihinde şunu yazdı:
> >
> > Unfortunately, it is present in 8.8.2 as well.
> >
> > On Mon, 3 Feb 2020 at 11:22, Ömer Sinan Ağacan 
> wrote:
> >>
> >> You should try with 8.8.2 which fixes a bug in the compacting GC
> (#17088).
> >>
> >> When debugging it's a good idea to use the latest minor release of your
> GHC
> >> version (8.8.2 in your case), as minor releases fix bugs and usually do
> not
> >> introduce new ones as they don't ship new features.
> >>
> >> If the problem still exists, unless you're interested in GHC hacking I
> think
> >> most productive use of the time would be to make the reproduer smaller,
> and
> >> collect as many data as possible, like which flags trigger/hide the bug.
> >>
> >> Some of the things you could check:
> >>
> >> - Build your program with `-dcore-lint -dstg-lint -dcmm-lint` and see
> if it
> >>   builds.
> >> - Build your program with `-debug` and run it, see if it crahes.
> >> - Build your program with `-debug` and run it with `+RTS -DS` and see
> if the
> >>   error message changes.
> >>
> >> But really you should try with 8.8.2 as first thing. It's possible that
> this is
> >> another manifestation of #17088.
> >>
> >> Ömer
> >>
> >> Harendra Kumar , 3 Şub 2020 Pzt, 01:26
> >> tarihinde şunu yazdı:
> >> >
> >> > Hi,
> >> >
> >> > While running a test-suite for the streaming library streamly I am
> encountering a crash which seems to happen at random places at different
> times. The common messages are:
> >> >
> >> > * Segmentation fault: 11
> >> > *  internal error: scavenge_mark_stack: unimplemented/strange closure
> type 24792696 @ 0x4200a623e0
> >> > * internal error: update_fwd: unknown/strange object  223743520
> >> >
> >> > and several other such messages. Prima facie this looks like the
> memory is getting corrupted/scribbled somehow. My first suspicion was that
> this could be a problem in the streamly library code. But I have stripped
> down the code to bare minimum and there is no C FFI code or no poking to
> memory pointers.
> >> >
> >> > My next suspicion was the hspec/quickcheck testing code that is being
> used in this test. I checked the hspec code to ensure that there is no C
> code/pointer poking in any of the code involved. But no luck there as well,
> still looking to further strip down that code.
> >> >
> >> > My suspicion now is moving more towards the GHC RTS. This issue only
> shows when the following conditions are met:
> >> >
> >> > * hspec "parallel" combinator is used to run tests in parallel
> >> > * streamly concurrent code is being tested which can create many
> threads
> >> > * The GHC heap size is restricted to a small size ~32MB using "-M32M"
> rts option.
> >> > * It is consistently seen with GHC 8.6.5 as well as GHC 8.8.1
> >> >
> >> > It never occurs when the heap size is not restricted. I have seen
> random crashes before as well with a "IO manager die" message, when using
> concurrent networking IO with streamly. Though earlier it was not easily
> reproducible, I stopped chasing it. But now it looks like that issue might
> also be a manifestation of the same underlying problem.
> >> >
> >> > My guess is it could be something in the RTS concurrency/threading
> related code. Let me know if the symptoms ring a bell or if you can point
> to something specific based on the symptoms. Also, what are the usual
> tools/methods/debugging aids/flags to debug such issues in

Re: Random crashes with memory corruption symptoms

[Harendra Kumar]

Unfortunately, it is present in 8.8.2 as well.

On Mon, 3 Feb 2020 at 11:22, Ömer Sinan Ağacan  wrote:

> You should try with 8.8.2 which fixes a bug in the compacting GC (#17088).
>
> When debugging it's a good idea to use the latest minor release of your GHC
> version (8.8.2 in your case), as minor releases fix bugs and usually do not
> introduce new ones as they don't ship new features.
>
> If the problem still exists, unless you're interested in GHC hacking I
> think
> most productive use of the time would be to make the reproduer smaller, and
> collect as many data as possible, like which flags trigger/hide the bug.
>
> Some of the things you could check:
>
> - Build your program with `-dcore-lint -dstg-lint -dcmm-lint` and see if it
>   builds.
> - Build your program with `-debug` and run it, see if it crahes.
> - Build your program with `-debug` and run it with `+RTS -DS` and see if
> the
>   error message changes.
>
> But really you should try with 8.8.2 as first thing. It's possible that
> this is
> another manifestation of #17088.
>
> Ömer
>
> Harendra Kumar , 3 Şub 2020 Pzt, 01:26
> tarihinde şunu yazdı:
> >
> > Hi,
> >
> > While running a test-suite for the streaming library streamly I am
> encountering a crash which seems to happen at random places at different
> times. The common messages are:
> >
> > * Segmentation fault: 11
> > *  internal error: scavenge_mark_stack: unimplemented/strange closure
> type 24792696 @ 0x4200a623e0
> > * internal error: update_fwd: unknown/strange object  223743520
> >
> > and several other such messages. Prima facie this looks like the memory
> is getting corrupted/scribbled somehow. My first suspicion was that this
> could be a problem in the streamly library code. But I have stripped down
> the code to bare minimum and there is no C FFI code or no poking to memory
> pointers.
> >
> > My next suspicion was the hspec/quickcheck testing code that is being
> used in this test. I checked the hspec code to ensure that there is no C
> code/pointer poking in any of the code involved. But no luck there as well,
> still looking to further strip down that code.
> >
> > My suspicion now is moving more towards the GHC RTS. This issue only
> shows when the following conditions are met:
> >
> > * hspec "parallel" combinator is used to run tests in parallel
> > * streamly concurrent code is being tested which can create many threads
> > * The GHC heap size is restricted to a small size ~32MB using "-M32M"
> rts option.
> > * It is consistently seen with GHC 8.6.5 as well as GHC 8.8.1
> >
> > It never occurs when the heap size is not restricted. I have seen random
> crashes before as well with a "IO manager die" message, when using
> concurrent networking IO with streamly. Though earlier it was not easily
> reproducible, I stopped chasing it. But now it looks like that issue might
> also be a manifestation of the same underlying problem.
> >
> > My guess is it could be something in the RTS concurrency/threading
> related code. Let me know if the symptoms ring a bell or if you can point
> to something specific based on the symptoms. Also, what are the usual
> tools/methods/debugging aids/flags to debug such issues in GHC? If not a
> GHC issue what are the possible ways in which such problem can be induced
> by application code?
> >
> > Meanwhile, I am also trying to simplify the reproducing code further to
> remove other factors as much as possible. The current code is at
> https://github.com/composewell/streamly on the ghc-segfault branch. Run
> "$ while true; do cabal run properties || break; done" in the shell and if
> you are lucky it may crash soon. The test code is in "test/Prop.hs" - here
> https://github.com/composewell/streamly/blob/ghc-segfault/test/Prop.hs .
> >
> > -harendra
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Random crashes with memory corruption symptoms

[Harendra Kumar]

Hi,

While running a test-suite for the streaming library streamly I am
encountering a crash which seems to happen at random places at different
times. The common messages are:

* Segmentation fault: 11
*  internal error: scavenge_mark_stack: unimplemented/strange closure type
24792696 @ 0x4200a623e0
* internal error: update_fwd: unknown/strange object  223743520

and several other such messages. Prima facie this looks like the memory is
getting corrupted/scribbled somehow. My first suspicion was that this could
be a problem in the streamly library code. But I have stripped down the
code to bare minimum and there is no C FFI code or no poking to memory
pointers.

My next suspicion was the hspec/quickcheck testing code that is being used
in this test. I checked the hspec code to ensure that there is no C
code/pointer poking in any of the code involved. But no luck there as well,
still looking to further strip down that code.

My suspicion now is moving more towards the GHC RTS. This issue only shows
when the following conditions are met:

* hspec "parallel" combinator is used to run tests in parallel
* streamly concurrent code is being tested which can create many threads
* The GHC heap size is restricted to a small size ~32MB using "-M32M"
rts option.
* It is consistently seen with GHC 8.6.5 as well as GHC 8.8.1

It never occurs when the heap size is not restricted. I have seen random
crashes before as well with a "IO manager die" message, when using
concurrent networking IO with streamly. Though earlier it was not easily
reproducible, I stopped chasing it. But now it looks like that issue might
also be a manifestation of the same underlying problem.

My guess is it could be something in the RTS concurrency/threading related
code. Let me know if the symptoms ring a bell or if you can point to
something specific based on the symptoms. Also, what are the usual
tools/methods/debugging aids/flags to debug such issues in GHC? If not a
GHC issue what are the possible ways in which such problem can be induced
by application code?

Meanwhile, I am also trying to simplify the reproducing code further to
remove other factors as much as possible. The current code is at
https://github.com/composewell/streamly on the ghc-segfault branch. Run "$
while true; do cabal run properties || break; done" in the shell and if you
are lucky it may crash soon. The test code is in "test/Prop.hs" - here
https://github.com/composewell/streamly/blob/ghc-segfault/test/Prop.hs .

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Hitting RTS bug on GHC 8.0.2

[Harendra Kumar]

Hit it only once. Cannot reproduce it after that. I will update if I hit it
again.

-harendra

On Tue, 30 Oct 2018 at 18:59, Simon Marlow  wrote:

> Looking at the code I can't see how that assertion could possibly fail.
> Is it reproducible?
>
> On Tue, 30 Oct 2018 at 08:38, Harendra Kumar 
> wrote:
>
>> Hi,
>>
>> I got the following crash in one of my CI tests (
>> https://travis-ci.org/composewell/streamly/jobs/448112763):
>>
>> test: internal error: RELEASE_LOCK: I do not own this lock:
>> rts/Messages.c 54
>> (GHC version 8.0.2 for x86_64_unknown_linux)
>> Please report this as a GHC bug:
>> http://www.haskell.org/ghc/reportabug
>>
>> I have hit this just once yet. Is this worth opening a ticket, given that
>> this is an older version of the compiler? Has something like been fixed
>> since then or might this be present in newer versions as well?
>>
>> -harendra
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Hitting RTS bug on GHC 8.0.2

[Harendra Kumar]

Hi,

I got the following crash in one of my CI tests (
https://travis-ci.org/composewell/streamly/jobs/448112763):

test: internal error: RELEASE_LOCK: I do not own this lock: rts/Messages.c
54
(GHC version 8.0.2 for x86_64_unknown_linux)
Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

I have hit this just once yet. Is this worth opening a ticket, given that
this is an older version of the compiler? Has something like been fixed
since then or might this be present in newer versions as well?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

clonetype

[Harendra Kumar]

Often, we need to create a newtype that is equivalent to a given type for
safety reasons. Using type  synonym is useless from type safety
perspective. With newtype, we have to add a "deriving" clause to it for
deriving the required instances, to make it practically useful.

Does it make sense, and is it possible to have something like a "clonetype"
that creates a new type and derives all the instances of the parent type as
well? It will be quite helpful in creating equivalent newtype synonyms
quickly. Almost always, I do not use a newtype where I should just because
of the inconvenience of deriving the instances. Ideally, we should just be
able to say something like:

clonetype MyString = String

and we are good to go.  What is the shortest possible way to achieve this
with currently available mechanisms, if any?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: DoAndIfThenElse

[Harendra Kumar]

Since I started programming in Haskell a few years ago I have been using
if-then-else in that manner without indentation and I never knew about this
extension. I thought this is how it works. It seems this is the default
now. But, I remember encountering an error in an older compiler version
once and then I figured the my style was accepted in newer compiler
versions only.

-harendra

On 9 February 2018 at 08:08, Brandon Allbery <allber...@gmail.com> wrote:

> Huh. I wonder if a section went missing; seems like none of the extensions
> that alter or relax layout are documented currently.
> (AlternativeLayoutRule, AlternativeLayoutRuleTransitional,
> DoAndIfThenElse, NondecreasingIndentation, RelaxedLayout)
>
> IIRC DoAndIfThenElse relaxes a condition implied by layout but that
> normally only matters in "do": that if you break it into multiple lines,
> the "then" and "else" must be indented farther than the "if" or layout will
> consider them distinct new expressions (and thereby syntax errors).
>
> On Thu, Feb 8, 2018 at 9:24 PM, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
>
>> Hi,
>>
>> I recently found a mention of DoAndIfThenElse extension somewhere. I
>> looked inside the ghc user guide and could not find any such extension.
>> Then I looked in the ghc man page, no mention. I googled and found a very
>> sparse references to it here and there. Then I tried using the extension
>> with ghc and ghc seems to accept it. What's the story behind this, why is
>> it not documented but accepted?
>>
>> thanks,
>> harendra
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>
>
> --
> brandon s allbery kf8nh   sine nomine
> associates
> allber...@gmail.com
> ballb...@sinenomine.net
> unix, openafs, kerberos, infrastructure, xmonad
> http://sinenomine.net
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

DoAndIfThenElse

[Harendra Kumar]

Hi,

I recently found a mention of DoAndIfThenElse extension somewhere. I looked
inside the ghc user guide and could not find any such extension. Then I
looked in the ghc man page, no mention. I googled and found a very sparse
references to it here and there. Then I tried using the extension with ghc
and ghc seems to accept it. What's the story behind this, why is it not
documented but accepted?

thanks,
harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [Haskell-cafe] rolling span and groupBy for lists

[Harendra Kumar]

On 6 February 2018 at 00:33, Sergiu Ivanov <siva...@colimite.fr> wrote:

> Thus quoth  Harendra Kumar  on Mon Feb 05 2018 at 18:30 (+0100):
> > Yes, Hayoo seems to be giving better results, I found more variants
> having
> > the behavior I want, it seems this variant is quite popular but still not
> > in any standard libraries.
> >
> > Interestingly the problem of too many choices and no standard one that
> can
> > be discovered applies to search engines as well. In this case there are
> > only two choices but still it is of the same nature. I knew about hayoo
> but
> > forgot to use it in this case. How much time should one spend on finding
> a
> > trivial function before giving up and making the choice to write their
> own?
> > I wish there was a standard, quick, good quality way of discovering what
> to
> > use.  It seems the Haskell ecosystem DNA encourages more and more
> > fragmentation rather than consolidation. I think the community/leaders
> > should acknowledge this problem and work on making things better in the
> > short/long run.
>
> A Single Liberal Unified Registry of Haskell Packages (SLUPR), an effort
> in this direction, has been recently announced:
>

Unfortunately, in my opinion, SLURP is taking things exactly in the
opposite direction. I was talking about the problem of choice above and
SLURP is giving even more choices  and therefore encouraging more
fragmentation. We should have just one good choice to stop wasting time and
energy finding the best choice among millions available. Everyone should
focus on making that one choice better rather spending energy in creating
their own alternatives. This is where the Haskell ecosystem philosophy
differs, it provides many choices in all aspects, it may be good in some
cases but not always. SLURP is a technology solution which exactly fits in
the same DNA. Technology can help us achieve the tasks that we set out to
do but technology cannot motivate and influence us in what we choose to do
and therefore ti cannot make the community focus on one goal - that
requires real people leadership. If we do not focus on one goal, even with
the best technology we may not succeed. Just my 2 cents.

-harendra



>
>
> > -harendra
> >
> > On 5 February 2018 at 22:02, Sergiu Ivanov <siva...@colimite.fr> wrote:
> >
> >> Hello Harendra,
> >>
> >> Thus quoth  Harendra Kumar  on Mon Feb 05 2018 at 16:43 (+0100):
> >> >
> >> > The irony is that theoretically you can find a Haskell package or
> >> > implementation of whatever you can imagine but quite often it takes
> more
> >> > time to discover it than writing your own.
> >>
> >> Sometimes Hayoo! helps me out in such situations:
> >>
> >>   http://hayoo.fh-wedel.de/?query=groupBy
> >>
> >> utility-ht shows up.
> >>
> >> --
> >> Sergiu
> >>
>
>
> --
> Sergiu
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: rolling span and groupBy for lists

[Harendra Kumar]

Yes, I did too :-) But there is a key difference in this case, all these
definitions are mathematically equivalent with identical semantics instead
of being some fuzzy subjective standards.

-harendra

On 5 February 2018 at 23:46, Brandon Allbery <allber...@gmail.com> wrote:

> Why do I suddenly catch a whiff of https://xkcd.com/927/ ?
>
> On Mon, Feb 5, 2018 at 1:13 PM, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
>
>> According to hayoo there seem to be 7 different implementations of this
>> same function. Yours is 8th and mine is 9th and other people may have more
>> not uploaded or maybe the ones that hayoo is not able to find. Does that
>> make a case for including this in some standard place?
>>
>> -harendra
>>
>> On 5 February 2018 at 12:22, Evan Laforge <qdun...@gmail.com> wrote:
>>
>>> I have my own list library with a bunch of things like this.  I think
>>> it's what most people do, and some upload them to hackage, e.g.
>>> utility-ht or the split package, or data-ordlist.
>>>
>>> Specifically, I think rollingGroupBy is what I call splitWith:
>>>
>>> -- | Split @xs@ before places where @f@ matches.
>>> --
>>> -- > split_with (==1) [1,2,1]
>>> -- > --> [[], [1, 2], [1]]
>>> split_with :: (a -> Bool) -> [a] -> NonNull [a]
>>> -- ^ output is non-null, and the contents are also, except the first
>>> one
>>>
>>> You can probably find something like this in 'split', or if not, that
>>> might be a good place to contribute it.
>>>
>>> I have a bunch of grouping functions too, which I use all the time, so
>>> if there's some kind of general list grouping package then maybe I
>>> could put them there.
>>>
>>> On the other hand, this sort of thing is pretty individual, so it
>>> doesn't seem so bad for each person to have their own local library.
>>> That way you know it fits your style.  Ultimately I think that's why
>>> none of the split functions made it into Data.List, every person has a
>>> slightly different idea of what it should be.
>>>
>>> On Sun, Feb 4, 2018 at 7:50 PM, Harendra Kumar <harendra.ku...@gmail.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > For a small problem, I was looking for a groupBy like function that
>>> groups
>>> > based on a predicate on successive elements but I could not find one. I
>>> > wrote these little functions for that purpose:
>>> >
>>> > -- | Like span, but with a predicate that compares two successive
>>> elements.
>>> > The
>>> > -- span ends when the two successive elements do not satisfy the
>>> predicate.
>>> > rollingSpan :: (a -> a -> Bool) -> [a] -> ([a], [a])
>>> > rollingSpan _ xs@[] = (xs, xs)
>>> > rollingSpan _ xs@[_] = (xs, [])
>>> > rollingSpan p (x1:xs@(x2:_))
>>> > | p x1 x2 =
>>> > let (ys, zs) = rollingSpan p xs
>>> > in (x1 : ys, zs)
>>> > | otherwise = ([x1], xs)
>>> >
>>> > -- | Like 'groupBy' but with a predicate that compares two successive
>>> > elements.
>>> > -- A group ends when two successive elements do not satisfy the
>>> predicate.
>>> > rollingGroupBy :: (a -> a -> Bool) -> [a] -> [[a]]
>>> > rollingGroupBy _ [] = []
>>> > rollingGroupBy cmp xs =
>>> > let (ys, zs) = rollingSpan cmp xs
>>> > in ys : rollingGroupBy cmp zs
>>> >
>>> > Are there any existing functions that serve this purpose or is there
>>> any
>>> > simpler way to achieve such functionality? If not, where is the right
>>> place
>>> > for these, if any. Can they be included in Data.List in base?
>>> >
>>> > Thanks,
>>> > Harendra
>>> >
>>> > ___
>>> > ghc-devs mailing list
>>> > ghc-devs@haskell.org
>>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>> >
>>>
>>
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>
>
> --
> brandon s allbery kf8nh   sine nomine
> associates
> allber...@gmail.com
> ballb...@sinenomine.net
> unix, openafs, kerberos, infrastructure, xmonad
> http://sinenomine.net
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: rolling span and groupBy for lists

[Harendra Kumar]

According to hayoo there seem to be 7 different implementations of this
same function. Yours is 8th and mine is 9th and other people may have more
not uploaded or maybe the ones that hayoo is not able to find. Does that
make a case for including this in some standard place?

-harendra

On 5 February 2018 at 12:22, Evan Laforge <qdun...@gmail.com> wrote:

> I have my own list library with a bunch of things like this.  I think
> it's what most people do, and some upload them to hackage, e.g.
> utility-ht or the split package, or data-ordlist.
>
> Specifically, I think rollingGroupBy is what I call splitWith:
>
> -- | Split @xs@ before places where @f@ matches.
> --
> -- > split_with (==1) [1,2,1]
> -- > --> [[], [1, 2], [1]]
> split_with :: (a -> Bool) -> [a] -> NonNull [a]
> -- ^ output is non-null, and the contents are also, except the first
> one
>
> You can probably find something like this in 'split', or if not, that
> might be a good place to contribute it.
>
> I have a bunch of grouping functions too, which I use all the time, so
> if there's some kind of general list grouping package then maybe I
> could put them there.
>
> On the other hand, this sort of thing is pretty individual, so it
> doesn't seem so bad for each person to have their own local library.
> That way you know it fits your style.  Ultimately I think that's why
> none of the split functions made it into Data.List, every person has a
> slightly different idea of what it should be.
>
> On Sun, Feb 4, 2018 at 7:50 PM, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
> > Hi,
> >
> > For a small problem, I was looking for a groupBy like function that
> groups
> > based on a predicate on successive elements but I could not find one. I
> > wrote these little functions for that purpose:
> >
> > -- | Like span, but with a predicate that compares two successive
> elements.
> > The
> > -- span ends when the two successive elements do not satisfy the
> predicate.
> > rollingSpan :: (a -> a -> Bool) -> [a] -> ([a], [a])
> > rollingSpan _ xs@[] = (xs, xs)
> > rollingSpan _ xs@[_] = (xs, [])
> > rollingSpan p (x1:xs@(x2:_))
> > | p x1 x2 =
> > let (ys, zs) = rollingSpan p xs
> > in (x1 : ys, zs)
> > | otherwise = ([x1], xs)
> >
> > -- | Like 'groupBy' but with a predicate that compares two successive
> > elements.
> > -- A group ends when two successive elements do not satisfy the
> predicate.
> > rollingGroupBy :: (a -> a -> Bool) -> [a] -> [[a]]
> > rollingGroupBy _ [] = []
> > rollingGroupBy cmp xs =
> > let (ys, zs) = rollingSpan cmp xs
> > in ys : rollingGroupBy cmp zs
> >
> > Are there any existing functions that serve this purpose or is there any
> > simpler way to achieve such functionality? If not, where is the right
> place
> > for these, if any. Can they be included in Data.List in base?
> >
> > Thanks,
> > Harendra
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [Haskell-cafe] rolling span and groupBy for lists

[Harendra Kumar]

Yes, Hayoo seems to be giving better results, I found more variants having
the behavior I want, it seems this variant is quite popular but still not
in any standard libraries.

Interestingly the problem of too many choices and no standard one that can
be discovered applies to search engines as well. In this case there are
only two choices but still it is of the same nature. I knew about hayoo but
forgot to use it in this case. How much time should one spend on finding a
trivial function before giving up and making the choice to write their own?
I wish there was a standard, quick, good quality way of discovering what to
use.  It seems the Haskell ecosystem DNA encourages more and more
fragmentation rather than consolidation. I think the community/leaders
should acknowledge this problem and work on making things better in the
short/long run.

-harendra

On 5 February 2018 at 22:02, Sergiu Ivanov <siva...@colimite.fr> wrote:

> Hello Harendra,
>
> Thus quoth  Harendra Kumar  on Mon Feb 05 2018 at 16:43 (+0100):
> >
> > The irony is that theoretically you can find a Haskell package or
> > implementation of whatever you can imagine but quite often it takes more
> > time to discover it than writing your own.
>
> Sometimes Hayoo! helps me out in such situations:
>
>   http://hayoo.fh-wedel.de/?query=groupBy
>
> utility-ht shows up.
>
> --
> Sergiu
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: rolling span and groupBy for lists

[Harendra Kumar]

On 5 February 2018 at 12:22, Evan Laforge <qdun...@gmail.com> wrote:

> I have my own list library with a bunch of things like this.  I think
> it's what most people do, and some upload them to hackage, e.g.
> utility-ht or the split package, or data-ordlist.
>

The irony is that theoretically you can find a Haskell package or
implementation of whatever you can imagine but quite often it takes more
time to discover it than writing your own. And then uploading what you
wrote to hackage compounds the problem. A hoogle search only shows the
groupBy in base, signature search also does not yield other results, it
seems hoogle does not cover those other packages. After writing my own and
spending quite a bit of time I could find two other similar implementations
of groupBy, one in "utility-ht" package and the other in
"data-list-sequences" but they don't turn up in hoogle search. It looks
like hoogle database should cover more packages, or maybe the search has
some issues. This state of affairs encourages people to write their own
rather than find and reuse stuff. My example in this email can be dismissed
as a small one but I think it is a larger problem.


> You can probably find something like this in 'split', or if not, that
> might be a good place to contribute it.
>

Yes, that is the fallback option I was considering, split seems to be the
most comprehensive of all such list related packages.


> I have a bunch of grouping functions too, which I use all the time, so
> if there's some kind of general list grouping package then maybe I
> could put them there.
>

It will be a great service to other Haskell explorers if we can consolidate
all such packages and make one standard package covering most use cases and
deprecate the other packages. Also it may be a good idea to have a see-also
or related packages kind of field in packages so that discovery is easy.


> On the other hand, this sort of thing is pretty individual, so it
> doesn't seem so bad for each person to have their own local library.
> That way you know it fits your style.  Ultimately I think that's why
> none of the split functions made it into Data.List, every person has a
> slightly different idea of what it should be.
>

I thought that rollingGroupBy would have been a better default option as it
can practically subsume the purpose of groupBy. groupBy in base is not well
documented, and intuitively many think it works the way rollingGroupBy
works i.e. compare two successive elements rather than comparing a fixed
element. See this stack overflow question
https://stackoverflow.com/questions/45654216/haskell-groupby-function-how-exactly-does-it-work
, I thought the same way. I guess if we compare two successive elements, by
transitive equality the existing groupBy implementation will practically
get covered by that, not strictly compatible but should serve all practical
purposes. That was the point why I was asking to consider having it in base
alongside groupBy. It seems more useful, general and intuitive than the
existing groupBy.

-harendra


>
> On Sun, Feb 4, 2018 at 7:50 PM, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
> > Hi,
> >
> > For a small problem, I was looking for a groupBy like function that
> groups
> > based on a predicate on successive elements but I could not find one. I
> > wrote these little functions for that purpose:
> >
> > -- | Like span, but with a predicate that compares two successive
> elements.
> > The
> > -- span ends when the two successive elements do not satisfy the
> predicate.
> > rollingSpan :: (a -> a -> Bool) -> [a] -> ([a], [a])
> > rollingSpan _ xs@[] = (xs, xs)
> > rollingSpan _ xs@[_] = (xs, [])
> > rollingSpan p (x1:xs@(x2:_))
> > | p x1 x2 =
> > let (ys, zs) = rollingSpan p xs
> > in (x1 : ys, zs)
> > | otherwise = ([x1], xs)
> >
> > -- | Like 'groupBy' but with a predicate that compares two successive
> > elements.
> > -- A group ends when two successive elements do not satisfy the
> predicate.
> > rollingGroupBy :: (a -> a -> Bool) -> [a] -> [[a]]
> > rollingGroupBy _ [] = []
> > rollingGroupBy cmp xs =
> > let (ys, zs) = rollingSpan cmp xs
> > in ys : rollingGroupBy cmp zs
> >
> > Are there any existing functions that serve this purpose or is there any
> > simpler way to achieve such functionality? If not, where is the right
> place
> > for these, if any. Can they be included in Data.List in base?
> >
> > Thanks,
> > Harendra
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: rolling span and groupBy for lists

[Harendra Kumar]

I was mainly asking if it makes sense to include these functions in
base/Data.List. Since the base package is maintained and ships along with
ghc, and the issues are also raised at ghc trac I thought this is the right
list. I am copying to librar...@haskell.org as well.

-harendra

On 5 February 2018 at 09:53, David Feuer <da...@well-typed.com> wrote:

> This is the wrong list. You probably meant to email haskell-cafe or
> perhaps librar...@haskell.org.
>
>
>
> David Feuer
> Well-Typed, LLP
>
>  Original message 
> From: Harendra Kumar <harendra.ku...@gmail.com>
> Date: 2/4/18 10:50 PM (GMT-05:00)
> To: ghc-devs@haskell.org
> Subject: rolling span and groupBy for lists
>
> Hi,
>
> For a small problem, I was looking for a groupBy like function that groups
> based on a predicate on successive elements but I could not find one. I
> wrote these little functions for that purpose:
>
> -- | Like span, but with a predicate that compares two successive elements.
> The
> -- span ends when the two successive elements do not satisfy the predicate.
> rollingSpan :: (a -> a -> Bool) -> [a] -> ([a], [a])
> rollingSpan _ xs@[] = (xs, xs)
> rollingSpan _ xs@[_] = (xs, [])
> rollingSpan p (x1:xs@(x2:_))
> | p x1 x2 =
> let (ys, zs) = rollingSpan p xs
> in (x1 : ys, zs)
> | otherwise = ([x1], xs)
>
> -- | Like 'groupBy' but with a predicate that compares two successive
> elements.
> -- A group ends when two successive elements do not satisfy the predicate.
> rollingGroupBy :: (a -> a -> Bool) -> [a] -> [[a]]
> rollingGroupBy _ [] = []
> rollingGroupBy cmp xs =
> let (ys, zs) = rollingSpan cmp xs
> in ys : rollingGroupBy cmp zs
>
> Are there any existing functions that serve this purpose or is there any
> simpler way to achieve such functionality? If not, where is the right place
> for these, if any. Can they be included in Data.List in base?
>
> Thanks,
> Harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

rolling span and groupBy for lists

[Harendra Kumar]

Hi,

For a small problem, I was looking for a groupBy like function that groups
based on a predicate on successive elements but I could not find one. I
wrote these little functions for that purpose:

-- | Like span, but with a predicate that compares two successive elements.
The
-- span ends when the two successive elements do not satisfy the predicate.
rollingSpan :: (a -> a -> Bool) -> [a] -> ([a], [a])
rollingSpan _ xs@[] = (xs, xs)
rollingSpan _ xs@[_] = (xs, [])
rollingSpan p (x1:xs@(x2:_))
| p x1 x2 =
let (ys, zs) = rollingSpan p xs
in (x1 : ys, zs)
| otherwise = ([x1], xs)

-- | Like 'groupBy' but with a predicate that compares two successive
elements.
-- A group ends when two successive elements do not satisfy the predicate.
rollingGroupBy :: (a -> a -> Bool) -> [a] -> [[a]]
rollingGroupBy _ [] = []
rollingGroupBy cmp xs =
let (ys, zs) = rollingSpan cmp xs
in ys : rollingGroupBy cmp zs

Are there any existing functions that serve this purpose or is there any
simpler way to achieve such functionality? If not, where is the right place
for these, if any. Can they be included in Data.List in base?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: How to load & parse an HI (interface) file?

[Harendra Kumar]

Also, "-ddump-hi" dumps the same information at compile time.

-harendra

On 3 December 2017 at 01:50, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> GHC has a "--show-iface" option which pretty prints the ".hi" file. Not
> sure if it works for your use-case but it may be easier to parse the text
> displayed by this option.
>
> -harendra
>
> On 2 December 2017 at 21:29, Saurabh Nanda <saurabhna...@gmail.com> wrote:
>
>> (GHC newbie alert -- is this the right mailing list for these kind of
>> questions?)
>>
>> I"m writing some code to figure out all the instances of particular
>> type-classes and after exploring a lot of options (hlint, haskell-src-exts,
>> annotations, doctests, etc), I realized that the compiler had already
>> figured it out and written it to disk for me!
>>
>> More digging led me to https://www.stackage.org/haddo
>> ck/lts-9.0/ghc-8.0.2/LoadIface.html#v:loadSrcInterface after which I got
>> stuck. How does one call this function? Specifically:
>>
>> * What is SDoc and how to construct a reasonable value for this argument?
>> * IsBootInterface would mostly be False, right?
>> * What does `Maybe FastString` represent and how does one construct it?
>> * Finally how does one evaluate the resulting monadic action to get
>> access to the underlying `ModIface`?
>>
>> -- Saurabh.
>>
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: How to load & parse an HI (interface) file?

[Harendra Kumar]

GHC has a "--show-iface" option which pretty prints the ".hi" file. Not
sure if it works for your use-case but it may be easier to parse the text
displayed by this option.

-harendra

On 2 December 2017 at 21:29, Saurabh Nanda  wrote:

> (GHC newbie alert -- is this the right mailing list for these kind of
> questions?)
>
> I"m writing some code to figure out all the instances of particular
> type-classes and after exploring a lot of options (hlint, haskell-src-exts,
> annotations, doctests, etc), I realized that the compiler had already
> figured it out and written it to disk for me!
>
> More digging led me to https://www.stackage.org/haddock/lts-9.0/ghc-8.0.2/
> LoadIface.html#v:loadSrcInterface after which I got stuck. How does one
> call this function? Specifically:
>
> * What is SDoc and how to construct a reasonable value for this argument?
> * IsBootInterface would mostly be False, right?
> * What does `Maybe FastString` represent and how does one construct it?
> * Finally how does one evaluate the resulting monadic action to get access
> to the underlying `ModIface`?
>
> -- Saurabh.
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Dynamically choosing the main function

[Harendra Kumar]

On 13 November 2017 at 03:47, Ben Gamari <b...@smart-cactus.org> wrote:

> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > On 12 November 2017 at 23:18, Ben Gamari <b...@smart-cactus.org> wrote:
> >
> >> In general it's not easy to get a reference to a an arbitrary function
> >> in an object file by it's source name. Unless the symbol is exported the
> >> compiler is free to optimize the function or even drop it altogether.
> >>
> >
> > I understand that. I was also wondering if there is way (some sort of
> > annotation, pragma or any other workaround) to keep the symbols around
> > without actually exporting them.  Exporting either clobbers the module
> > interface from user point of view or the modules need a wrapper to avoid
> > exporting such symbols in user exposed modules. I would also expect an
> > option to remove the effect of any such annotation so that the production
> > build is not under optimized because of this. This is in fact one of the
> > problems that I have been facing in general. I do not know of a good way
> to
> > hide symbols from users but use them in tests and benchmarks during dev.
> > What is the GHC recommended way to achieve this?
> >
> The usual way of dealing with this is to expose the symbols from a
> `.Internal` module.
>

That's what I meant when I referred to a wrapper module above. However, it
requires you to create two modules even if one extra symbol is needed for
dev purposes, a lot of boilerplate for a simple thing. Also, the symbols
are anyway exposed to the users, we just ask the users to not look at
those. I am wondering if using a preprocessor (CPP) conditional around the
optional exports is a better way then, you will have to build differently
but you won't have to create another module and expose extra symbols?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Dynamically choosing the main function

[Harendra Kumar]

On 13 November 2017 at 00:29, Saurabh Nanda  wrote:

> > In fact I have been using dlopen to implement Haskell plugins but did
> not think of using it for this use case.
>
> Absolutely OT. Is this technique different from what facebook/simonmar
> have recently open-sourced? Please blog about it.
>

 Take a look at this: http://hackage.haskell.org/package/plugins

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Dynamically choosing the main function

[Harendra Kumar]

On 12 November 2017 at 23:18, Ben Gamari <b...@smart-cactus.org> wrote:

> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > Hi,
> >
> > GHC allows choosing a main function at link time using the "-main-is"
> > option. I was wondering if there is a possibility to choose the main
> > function at runtime. Or even better, if something equivalent to "ghc -e"
> > is somehow possible in a linked binary executable. If not, are there any
> > plans to achieve something like that in future? Are there any
> theoretical,
> > practical obstacles to that?
> >
> In general it's not easy to get a reference to a an arbitrary function
> in an object file by it's source name. Unless the symbol is exported the
> compiler is free to optimize the function or even drop it altogether.
>

I understand that. I was also wondering if there is way (some sort of
annotation, pragma or any other workaround) to keep the symbols around
without actually exporting them.  Exporting either clobbers the module
interface from user point of view or the modules need a wrapper to avoid
exporting such symbols in user exposed modules. I would also expect an
option to remove the effect of any such annotation so that the production
build is not under optimized because of this. This is in fact one of the
problems that I have been facing in general. I do not know of a good way to
hide symbols from users but use them in tests and benchmarks during dev.
What is the GHC recommended way to achieve this?


> If you can guarantee that the functions you are about are exported then
> in principle you could probably do what you ask from a library. Just
> `dlopen` the executable (which would need to be compiled as a position
> independent exectuable), and dlsym the appropriately mangled name.
>

Yeah that is a good option for dynamically linked executables. In fact I
have been using dlopen to implement Haskell plugins but did not think of
using it for this use case. Thanks for reminding me. In a static
executable, does it make sense to make the RTS choose an exported symbol as
entry point using an RTS option?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Dynamically choosing the main function

[Harendra Kumar]

For example, I can easily define entry points for arbitrary functions. At
run time I can choose which entry point to call and benchmark that entry
point using existing performance benchmarking tools like the very powerful
and comprehensive "perf" tool on linux. This provides a very powerful way
to analyze the performance for free and in a convenient manner with little
instrumentation. I do not want the main entry point to be clobbered for
such things. Ideally I would just use a simple annotation for any arbitrary
function to convert it into an entry point that can be chosen at runtime. I
am just giving a general idea here without going into specifics.

-harendra

On 12 November 2017 at 21:58, Shea Levy <s...@shealevy.com> wrote:

> What would you be able to achieve with this that you couldn't achieve
> with branching in a fixed custom main function?
>
> Thanks,
> Shea
>
> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > Hi,
> >
> > GHC allows choosing a main function at link time using the "-main-is"
> > option. I was wondering if there is a possibility to choose the main
> > function at runtime. Or even better, if something equivalent to "ghc -e"
> > is somehow possible in a linked binary executable. If not, are there any
> > plans to achieve something like that in future? Are there any
> theoretical,
> > practical obstacles to that?
> >
> > Thanks,
> > Harendra
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Dynamically choosing the main function

[Harendra Kumar]

Hi,

GHC allows choosing a main function at link time using the "-main-is"
option. I was wondering if there is a possibility to choose the main
function at runtime. Or even better, if something equivalent to "ghc -e"
is somehow possible in a linked binary executable. If not, are there any
plans to achieve something like that in future? Are there any theoretical,
practical obstacles to that?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.2 GC time stats going negative

[Harendra Kumar]

Ben,

I could reproduce it even after removing all FFI calls. I created a ghc
trac ticket, you can find it here:
https://ghc.haskell.org/trac/ghc/ticket/14445. Let me know if any more
information is needed or if explanation about the code/repo is required.

Thanks,
Harendra


On 10 November 2017 at 00:12, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> Hi Ben,
>
> Yes, there are FFI calls between the two getRTSStats calls. FFI calls are
> C APIs to get some OS stats like clock time or cpu time etc. Now I know
> that this is not expected, let me try to minimize the example and see where
> it goes. I will try removing the FFI calls as well.
>
> -harendra
>
> On 9 November 2017 at 23:49, Ben Gamari <b...@well-typed.com> wrote:
>
>> Harendra Kumar <harendra.ku...@gmail.com> writes:
>>
>> > Hi,
>> >
>> > I am trying to use the mutator_cpu_ns and mutator_elapsed_ns from GHC
>> 8.2
>> > RTSStats structure retrieved using getRTSStats API documented here -
>> > https://hackage.haskell.org/package/base-4.10.0.0/docs/GHC-Stats.html .
>> >
>> > I am seeing that the values of these stats retrieved about 30 ms later
>> are
>> > lower than the values retrieved earlier. Sometimes even after 100 ms the
>> > values decrease from the previous ones. However, as the time duration
>> > between two calls increase this becomes less and less likely.
>> >
>> > Is this an expected behavior or am I using it incorrectly? If it is
>> > expected, why does it happen, what is the underlying mechanism that
>> makes
>> > it happen?
>> >
>> Hmm, interesting. I don't believe this should happen and I do know of
>> bugs in the RTS's time accounting at exit-time (manifesting in profiling
>> issues, e.g. #14257) however what you describe doesn't seem to fit that
>> description. Do you have a simple repro? Is there anything special about
>> your program (e.g. lots of FFI calls)?
>>
>> Cheers,
>>
>> - Ben
>>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.2 GC time stats going negative

[Harendra Kumar]

Hi Ben,

Yes, there are FFI calls between the two getRTSStats calls. FFI calls are C
APIs to get some OS stats like clock time or cpu time etc. Now I know that
this is not expected, let me try to minimize the example and see where it
goes. I will try removing the FFI calls as well.

-harendra

On 9 November 2017 at 23:49, Ben Gamari <b...@well-typed.com> wrote:

> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > Hi,
> >
> > I am trying to use the mutator_cpu_ns and mutator_elapsed_ns from GHC 8.2
> > RTSStats structure retrieved using getRTSStats API documented here -
> > https://hackage.haskell.org/package/base-4.10.0.0/docs/GHC-Stats.html .
> >
> > I am seeing that the values of these stats retrieved about 30 ms later
> are
> > lower than the values retrieved earlier. Sometimes even after 100 ms the
> > values decrease from the previous ones. However, as the time duration
> > between two calls increase this becomes less and less likely.
> >
> > Is this an expected behavior or am I using it incorrectly? If it is
> > expected, why does it happen, what is the underlying mechanism that makes
> > it happen?
> >
> Hmm, interesting. I don't believe this should happen and I do know of
> bugs in the RTS's time accounting at exit-time (manifesting in profiling
> issues, e.g. #14257) however what you describe doesn't seem to fit that
> description. Do you have a simple repro? Is there anything special about
> your program (e.g. lots of FFI calls)?
>
> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC 8.2 GC time stats going negative

[Harendra Kumar]

Hi,

I am trying to use the mutator_cpu_ns and mutator_elapsed_ns from GHC 8.2
RTSStats structure retrieved using getRTSStats API documented here -
https://hackage.haskell.org/package/base-4.10.0.0/docs/GHC-Stats.html .

I am seeing that the values of these stats retrieved about 30 ms later are
lower than the values retrieved earlier. Sometimes even after 100 ms the
values decrease from the previous ones. However, as the time duration
between two calls increase this becomes less and less likely.

Is this an expected behavior or am I using it incorrectly? If it is
expected, why does it happen, what is the underlying mechanism that makes
it happen?

Thanks,
harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.2.1 release commit

[Harendra Kumar]

It seems I had only the master branch in my tree. A git fetch showed all
the branches including 8.2.1.

-harendra

On 11 September 2017 at 11:13, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> Hi,
>
> In the GHC git repo how do I figure out which commit belongs to release
> 8.2.1. I cannot find 8.2.1 in "git tag" output. I tried "git log" and
> searching for 8.2.1 but there seems to be no definitive comment marking
> 8.2.1.
>
> -harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC 8.2.1 release commit

[Harendra Kumar]

Hi,

In the GHC git repo how do I figure out which commit belongs to release
8.2.1. I cannot find 8.2.1 in "git tag" output. I tried "git log" and
searching for 8.2.1 but there seems to be no definitive comment marking
8.2.1.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Semigroup repeat (base package)

[Harendra Kumar]

On 11 September 2017 at 02:46, Wolfgang Jeltsch 
wrote:

> Am Sonntag, den 10.09.2017, 10:39 +0200 schrieb Herbert Valerio Riedel:
> > What you seem to be searching for looks more like what we know as
> > `cycle :: [a] -> [a]`, and in fact there is its generalisation at
> >
> > http://hackage.haskell.org/package/base-4.10.0.0/docs/
> Data-Semigroup.html#v:cycle1
>
> Why is this function called cycle1, not cycle? What does the “1” stand
> for?


I guess this is not named "cycle" to avoid conflict with "Data.List.cycle".
I was also wondering why it is "cycle1" instead of, say "scycle". It can be
thought of as cycling just one value instead of cycling a list in case of
"Data.List.cycle". I Just made up this explanation, original writers would
know better. I would prefer "scycle" which is consistent with other
functions in this package like "stimes", cycle1 sounds a bit random at
first look.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Semigroup repeat (base package)

[Harendra Kumar]

Indeed it is cycle1, sorry to have missed it. It will be easier to spot it
if it close to the stimes* functions in the beginning of the docs. It is
placed too far down below even after Monoid re-exports.

-harendra

On 10 September 2017 at 14:09, Herbert Valerio Riedel <hvrie...@gmail.com>
wrote:

> Hi,
>
> On Sun, Sep 10, 2017 at 9:24 AM, Harendra Kumar
> <harendra.ku...@gmail.com> wrote:
> > I could not find a function that repeats a value using a semigroup
> append. I
> > am looking for something like this:
> >
> > srepeat :: Semigroup a => a -> a
> > srepeat x = xs where xs = x <> xs
> >
> > Is it already available somewhere? Does it make sense to add it to
> > Data.Semigroup?
>
> What you seem to be searching for looks more like what we know as
> `cycle :: [a] -> [a]`, and in fact there is its generalisation at
>
> http://hackage.haskell.org/package/base-4.10.0.0/docs/
> Data-Semigroup.html#v:cycle1
>
> hth
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Semigroup repeat (base package)

[Harendra Kumar]

Hi,

I am sending this question here since base ships with ghc, let me know if
this is not the right forum for this.

I could not find a function that repeats a value using a semigroup append.
I am looking for something like this:

srepeat :: Semigroup a => a -> a
srepeat x = xs where xs = x <> xs

Is it already available somewhere? Does it make sense to add it to
Data.Semigroup?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance degradation when factoring out common code

[Harendra Kumar]

Ok, I filed a ticket for the inlining issue as well -
https://ghc.haskell.org/trac/ghc/ticket/14211. The reproduction test case
is in the same repo on the "inlining-issue" branch, here -
https://github.com/harendra-kumar/ghc-perf/tree/inlining-issue .

Performance with manually inlining a function is more than 10% faster
compared to factoring out code and using INLINE pragma.

stack bench for compiler inlined code

time 46.71 ms   (45.53 ms .. 47.79 ms)

stack bench --flag ghc-perf:manual for manually inlined code

time 39.46 ms   (38.92 ms .. 39.94 ms)


-harendra

On 9 September 2017 at 14:08, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> The code is at: https://github.com/harendra-kumar/asyncly. The benchmark
> code is in "benchmark/Main.hs".  The relevant function is "asyncly_basic".
>
> If you want to run it, you can use the following steps to reproduce the
> behavior I reported below:
>
> 1) Run "stack build"
> 2) Run "stack runghc benchmark/Main.hs" for runghc figures
> 3) Run "stack ghc benchmark/Main.hs && benchmark/Main" to compile and run
> normally
> 4) Run "stack ghc -- -O2 benchmark/Main.hs && benchmark/Main" to compile
> and run with -O2 flag
>
> Just look at the first benchmark (asyncly-serial), you can comment out all
> others if you want to. Note that the library gets compiled without any
> optimization flags (see the ghc options in the cabal file). So what we are
> seeing here is just the effect of -O2 on compiling benchmarks/Main.hs.
>
> I am also trying to isolate the problem to a minimal case. I tried
> removing all the INLINE pragmas in the library to make sure that I am not
> screwing it up by asking the compiler to inline aggressively, but that does
> not seem to make any difference to the situation. Let me know if you need
> any information from me or help in running it.
>
> There are three issues that I am trying to get answers for:
>
> 1) Why runghc is faster? It means that there is a possibility for the
> program to run as fast as runghc runs it. How do I get that performance or
> an explanation of it?
>
> 2) Why -O1/O2 degrades performance so much by 4-5x.
>
> 3) The third one is the original problem that I posted in this thread,
> compiler is unable to match manual inlining. It is possible that this is an
> issue only when -O1/O2 is used and not when -O0 is used.
>
> Thanks for the help.
>
> -harendra
>
>
> On 9 September 2017 at 13:30, Matthew Pickering <
> matthewtpicker...@gmail.com> wrote:
>
>> Do you have the code?
>>
>> On Sat, Sep 9, 2017 at 6:05 AM, Harendra Kumar <harendra.ku...@gmail.com>
>> wrote:
>> > While trying to come up with a minimal example I discovered one more
>> > puzzling thing. runghc is fastest, ghc is slower, ghc with optimization
>> is
>> > slowest. This is completely reverse of the expected order.
>> >
>> > ghc -O1 (-O2 is similar):
>> >
>> > time 15.23 ms   (14.72 ms .. 15.73 ms)
>> >
>> > ghc -O0:
>> >
>> > time 3.612 ms   (3.548 ms .. 3.728 ms)
>> >
>> > runghc:
>> >
>> > time 2.250 ms   (2.156 ms .. 2.348 ms)
>> >
>> >
>> > I am grokking it further. Any pointers will be helpful. I understand
>> that
>> > -O2 can sometimes be slower e.g. aggressive inlining can sometimes be
>> > counterproductive. But 4x variation is a lot and this is the case with
>> -O1
>> > as well which should be relatively safer than -O2 in general. Worst of
>> all
>> > runghc is significantly faster than ghc. What's going on?
>> >
>> > -harendra
>> >
>> >
>> > On 8 September 2017 at 18:49, Harendra Kumar <harendra.ku...@gmail.com>
>> > wrote:
>> >>
>> >> I will try creating a minimal example and open a ticket for the
>> inlining
>> >> problem, the one I am sure about.
>> >>
>> >> -harendra
>> >>
>> >> On 8 September 2017 at 18:35, Simon Peyton Jones <
>> simo...@microsoft.com>
>> >> wrote:
>> >>>
>> >>> I know that this is not an easy request, but can either of you
>> produce a
>> >>> small example that demonstrates your problem?   If so, please open a
>> ticket.
>> >>>
>> >>>
>> >>>
>> >>> I don’t like hearing about people having to use trial and error  with
>> >>> INLINE or SPECIALISE pragmas.  But I can’t ev

Re: Performance degradation when factoring out common code

[Harendra Kumar]

I could pinpoint one part of the problem. Please see the ticket:
https://ghc.haskell.org/trac/ghc/ticket/14208. Here is the description that
I wrote in the ticket:

In this particular case -O2 is 2x slower than -O0 and -O0 is 2x slower than
runghc. Please see the github repo: ​
https://github.com/harendra-kumar/ghc-perf to reproduce the issue. Readme
file in the repo has instructions to reproduce.

The issue seems to occur when the code is placed in a different module.
When all the code is in the same module the problem does not occur. In that
case -O2 is faster than -O0. However, when the code is split into two
modules the performance gets inverted.

Also, it does not occur always, when I tried to change the code to make it
simpler for repro the problem did not occur.


-harendra

On 9 September 2017 at 14:08, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> The code is at: https://github.com/harendra-kumar/asyncly. The benchmark
> code is in "benchmark/Main.hs".  The relevant function is "asyncly_basic".
>
> If you want to run it, you can use the following steps to reproduce the
> behavior I reported below:
>
> 1) Run "stack build"
> 2) Run "stack runghc benchmark/Main.hs" for runghc figures
> 3) Run "stack ghc benchmark/Main.hs && benchmark/Main" to compile and run
> normally
> 4) Run "stack ghc -- -O2 benchmark/Main.hs && benchmark/Main" to compile
> and run with -O2 flag
>
> Just look at the first benchmark (asyncly-serial), you can comment out all
> others if you want to. Note that the library gets compiled without any
> optimization flags (see the ghc options in the cabal file). So what we are
> seeing here is just the effect of -O2 on compiling benchmarks/Main.hs.
>
> I am also trying to isolate the problem to a minimal case. I tried
> removing all the INLINE pragmas in the library to make sure that I am not
> screwing it up by asking the compiler to inline aggressively, but that does
> not seem to make any difference to the situation. Let me know if you need
> any information from me or help in running it.
>
> There are three issues that I am trying to get answers for:
>
> 1) Why runghc is faster? It means that there is a possibility for the
> program to run as fast as runghc runs it. How do I get that performance or
> an explanation of it?
>
> 2) Why -O1/O2 degrades performance so much by 4-5x.
>
> 3) The third one is the original problem that I posted in this thread,
> compiler is unable to match manual inlining. It is possible that this is an
> issue only when -O1/O2 is used and not when -O0 is used.
>
> Thanks for the help.
>
> -harendra
>
>
> On 9 September 2017 at 13:30, Matthew Pickering <
> matthewtpicker...@gmail.com> wrote:
>
>> Do you have the code?
>>
>> On Sat, Sep 9, 2017 at 6:05 AM, Harendra Kumar <harendra.ku...@gmail.com>
>> wrote:
>> > While trying to come up with a minimal example I discovered one more
>> > puzzling thing. runghc is fastest, ghc is slower, ghc with optimization
>> is
>> > slowest. This is completely reverse of the expected order.
>> >
>> > ghc -O1 (-O2 is similar):
>> >
>> > time 15.23 ms   (14.72 ms .. 15.73 ms)
>> >
>> > ghc -O0:
>> >
>> > time 3.612 ms   (3.548 ms .. 3.728 ms)
>> >
>> > runghc:
>> >
>> > time 2.250 ms   (2.156 ms .. 2.348 ms)
>> >
>> >
>> > I am grokking it further. Any pointers will be helpful. I understand
>> that
>> > -O2 can sometimes be slower e.g. aggressive inlining can sometimes be
>> > counterproductive. But 4x variation is a lot and this is the case with
>> -O1
>> > as well which should be relatively safer than -O2 in general. Worst of
>> all
>> > runghc is significantly faster than ghc. What's going on?
>> >
>> > -harendra
>> >
>> >
>> > On 8 September 2017 at 18:49, Harendra Kumar <harendra.ku...@gmail.com>
>> > wrote:
>> >>
>> >> I will try creating a minimal example and open a ticket for the
>> inlining
>> >> problem, the one I am sure about.
>> >>
>> >> -harendra
>> >>
>> >> On 8 September 2017 at 18:35, Simon Peyton Jones <
>> simo...@microsoft.com>
>> >> wrote:
>> >>>
>> >>> I know that this is not an easy request, but can either of you
>> produce a
>> >>> small example that demonstrates your problem?   If so, please open a
>> ticket.
>> >>>
>> >>>

Re: Performance degradation when factoring out common code

[Harendra Kumar]

The code is at: https://github.com/harendra-kumar/asyncly. The benchmark
code is in "benchmark/Main.hs".  The relevant function is "asyncly_basic".

If you want to run it, you can use the following steps to reproduce the
behavior I reported below:

1) Run "stack build"
2) Run "stack runghc benchmark/Main.hs" for runghc figures
3) Run "stack ghc benchmark/Main.hs && benchmark/Main" to compile and run
normally
4) Run "stack ghc -- -O2 benchmark/Main.hs && benchmark/Main" to compile
and run with -O2 flag

Just look at the first benchmark (asyncly-serial), you can comment out all
others if you want to. Note that the library gets compiled without any
optimization flags (see the ghc options in the cabal file). So what we are
seeing here is just the effect of -O2 on compiling benchmarks/Main.hs.

I am also trying to isolate the problem to a minimal case. I tried removing
all the INLINE pragmas in the library to make sure that I am not screwing
it up by asking the compiler to inline aggressively, but that does not seem
to make any difference to the situation. Let me know if you need any
information from me or help in running it.

There are three issues that I am trying to get answers for:

1) Why runghc is faster? It means that there is a possibility for the
program to run as fast as runghc runs it. How do I get that performance or
an explanation of it?

2) Why -O1/O2 degrades performance so much by 4-5x.

3) The third one is the original problem that I posted in this thread,
compiler is unable to match manual inlining. It is possible that this is an
issue only when -O1/O2 is used and not when -O0 is used.

Thanks for the help.

-harendra

On 9 September 2017 at 13:30, Matthew Pickering <matthewtpicker...@gmail.com
> wrote:

> Do you have the code?
>
> On Sat, Sep 9, 2017 at 6:05 AM, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
> > While trying to come up with a minimal example I discovered one more
> > puzzling thing. runghc is fastest, ghc is slower, ghc with optimization
> is
> > slowest. This is completely reverse of the expected order.
> >
> > ghc -O1 (-O2 is similar):
> >
> > time 15.23 ms   (14.72 ms .. 15.73 ms)
> >
> > ghc -O0:
> >
> > time 3.612 ms   (3.548 ms .. 3.728 ms)
> >
> > runghc:
> >
> > time 2.250 ms   (2.156 ms .. 2.348 ms)
> >
> >
> > I am grokking it further. Any pointers will be helpful. I understand that
> > -O2 can sometimes be slower e.g. aggressive inlining can sometimes be
> > counterproductive. But 4x variation is a lot and this is the case with
> -O1
> > as well which should be relatively safer than -O2 in general. Worst of
> all
> > runghc is significantly faster than ghc. What's going on?
> >
> > -harendra
> >
> >
> > On 8 September 2017 at 18:49, Harendra Kumar <harendra.ku...@gmail.com>
> > wrote:
> >>
> >> I will try creating a minimal example and open a ticket for the inlining
> >> problem, the one I am sure about.
> >>
> >> -harendra
> >>
> >> On 8 September 2017 at 18:35, Simon Peyton Jones <simo...@microsoft.com
> >
> >> wrote:
> >>>
> >>> I know that this is not an easy request, but can either of you produce
> a
> >>> small example that demonstrates your problem?   If so, please open a
> ticket.
> >>>
> >>>
> >>>
> >>> I don’t like hearing about people having to use trial and error  with
> >>> INLINE or SPECIALISE pragmas.  But I can’t even begin to solve the
> problem
> >>> unless I can reproduce it.
> >>>
> >>>
> >>>
> >>> Simon
> >>>
> >>>
> >>>
> >>> From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of
> >>> Harendra Kumar
> >>> Sent: 08 September 2017 13:50
> >>> To: Mikolaj Konarski <mikolaj.konar...@gmail.com>
> >>> Cc: ghc-devs@haskell.org
> >>> Subject: Re: Performance degradation when factoring out common code
> >>>
> >>>
> >>>
> >>> I should also point out that I saw performance improvements by manually
> >>> factoring out and propagating some common expressions to outer loops in
> >>> performance sensitive paths. Now I have made this a habit to do this
> >>> manually. Not sure if something like this has also been fixed with that
> >>> ticket or some other ticket.
> >>>
> >>>
> >>>
> >>> -harendra
> >>>
> 

Re: Performance degradation when factoring out common code

[Harendra Kumar]

While trying to come up with a minimal example I discovered one more
puzzling thing. runghc is fastest, ghc is slower, ghc with optimization is
slowest. This is completely reverse of the expected order.

ghc -O1 (-O2 is similar):

time 15.23 ms   (14.72 ms .. 15.73 ms)

ghc -O0:

time 3.612 ms   (3.548 ms .. 3.728 ms)

runghc:

time 2.250 ms   (2.156 ms .. 2.348 ms)


I am grokking it further. Any pointers will be helpful. I understand that
-O2 can sometimes be slower e.g. aggressive inlining can sometimes be
counterproductive. But 4x variation is a lot and this is the case with -O1
as well which should be relatively safer than -O2 in general. Worst of all
runghc is significantly faster than ghc. What's going on?

-harendra

On 8 September 2017 at 18:49, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> I will try creating a minimal example and open a ticket for the inlining
> problem, the one I am sure about.
>
> -harendra
>
> On 8 September 2017 at 18:35, Simon Peyton Jones <simo...@microsoft.com>
> wrote:
>
>> *I know that this is not an easy request*, but can either of you produce
>> a small example that demonstrates your problem?   If so, please open a
>> ticket.
>>
>>
>>
>> I don’t like hearing about people having to use trial and error  with
>> INLINE or SPECIALISE pragmas.  But I can’t even begin to solve the problem
>> unless I can reproduce it.
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org] *On Behalf Of 
>> *Harendra
>> Kumar
>> *Sent:* 08 September 2017 13:50
>> *To:* Mikolaj Konarski <mikolaj.konar...@gmail.com>
>> *Cc:* ghc-devs@haskell.org
>> *Subject:* Re: Performance degradation when factoring out common code
>>
>>
>>
>> I should also point out that I saw performance improvements by manually
>> factoring out and propagating some common expressions to outer loops in
>> performance sensitive paths. Now I have made this a habit to do this
>> manually. Not sure if something like this has also been fixed with that
>> ticket or some other ticket.
>>
>>
>>
>> -harendra
>>
>>
>>
>> On 8 September 2017 at 17:34, Harendra Kumar <harendra.ku...@gmail.com>
>> wrote:
>>
>> Thanks Mikolaj! I have seen some surprising behavior quite a few times
>> recently and I was wondering whether GHC should do better. In one case I
>> had to use SPECIALIZE very aggressively, in another version of the same
>> code it worked well without that. I have been doing a lot of trial and
>> error with the INLINE/NOINLINE pragmas to figure out what the right
>> combination is. Sometimes it just feels like black magic, because I cannot
>> find a rationale to explain the behavior. I am not sure if there are any
>> more such problems lurking in, perhaps this is an area where some
>> improvement looks possible.
>>
>>
>>
>> -harendra
>>
>>
>>
>>
>>
>> On 8 September 2017 at 17:10, Mikolaj Konarski <
>> mikolaj.konar...@gmail.com> wrote:
>>
>> Hello,
>>
>> I've had a similar problem that's been fixed in 8.2.1:
>>
>> https://ghc.haskell.org/trac/ghc/ticket/12603
>>
>> You can also use some extreme global flags, such as
>>
>> ghc-options: -fexpose-all-unfoldings -fspecialise-aggressively
>>
>> to get most the GHC subtlety and shyness out of the way
>> when experimenting.
>>
>> Good luck
>> Mikolaj
>>
>>
>>
>>
>> On Fri, Sep 8, 2017 at 11:21 AM, Harendra Kumar
>> <harendra.ku...@gmail.com> wrote:
>> > Hi,
>> >
>> > I have this code snippet for the bind implementation of a Monad:
>> >
>> > AsyncT m >>= f = AsyncT $ \_ stp yld ->
>> > let run x = (runAsyncT x) Nothing stp yld
>> > yield a _ Nothing  = run $ f a
>> > yield a _ (Just r) = run $ f a <> (r >>= f)
>> > in m Nothing stp yield
>> >
>> > I want to have multiple versions of this implementation parameterized
>> by a
>> > function, like this:
>> >
>> > bindWith k (AsyncT m) f = AsyncT $ \_ stp yld ->
>> > let run x = (runAsyncT x) Nothing stp yld
>> > yield a _ Nothing  = run $ f a
>> > yield a _ (Just r) = run $ f a `k` (bindWith k r f)
>> > in m Nothing stp yield
>> >
>> > And then the bind function becomes:
>> >
>> > (>>=) = bindWith (<>)
>

Re: Performance degradation when factoring out common code

[Harendra Kumar]

I will try creating a minimal example and open a ticket for the inlining
problem, the one I am sure about.

-harendra

On 8 September 2017 at 18:35, Simon Peyton Jones <simo...@microsoft.com>
wrote:

> *I know that this is not an easy request*, but can either of you produce
> a small example that demonstrates your problem?   If so, please open a
> ticket.
>
>
>
> I don’t like hearing about people having to use trial and error  with
> INLINE or SPECIALISE pragmas.  But I can’t even begin to solve the problem
> unless I can reproduce it.
>
>
>
> Simon
>
>
>
> *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org] *On Behalf Of *Harendra
> Kumar
> *Sent:* 08 September 2017 13:50
> *To:* Mikolaj Konarski <mikolaj.konar...@gmail.com>
> *Cc:* ghc-devs@haskell.org
> *Subject:* Re: Performance degradation when factoring out common code
>
>
>
> I should also point out that I saw performance improvements by manually
> factoring out and propagating some common expressions to outer loops in
> performance sensitive paths. Now I have made this a habit to do this
> manually. Not sure if something like this has also been fixed with that
> ticket or some other ticket.
>
>
>
> -harendra
>
>
>
> On 8 September 2017 at 17:34, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
>
> Thanks Mikolaj! I have seen some surprising behavior quite a few times
> recently and I was wondering whether GHC should do better. In one case I
> had to use SPECIALIZE very aggressively, in another version of the same
> code it worked well without that. I have been doing a lot of trial and
> error with the INLINE/NOINLINE pragmas to figure out what the right
> combination is. Sometimes it just feels like black magic, because I cannot
> find a rationale to explain the behavior. I am not sure if there are any
> more such problems lurking in, perhaps this is an area where some
> improvement looks possible.
>
>
>
> -harendra
>
>
>
>
>
> On 8 September 2017 at 17:10, Mikolaj Konarski <mikolaj.konar...@gmail.com>
> wrote:
>
> Hello,
>
> I've had a similar problem that's been fixed in 8.2.1:
>
> https://ghc.haskell.org/trac/ghc/ticket/12603
>
> You can also use some extreme global flags, such as
>
> ghc-options: -fexpose-all-unfoldings -fspecialise-aggressively
>
> to get most the GHC subtlety and shyness out of the way
> when experimenting.
>
> Good luck
> Mikolaj
>
>
>
>
> On Fri, Sep 8, 2017 at 11:21 AM, Harendra Kumar
> <harendra.ku...@gmail.com> wrote:
> > Hi,
> >
> > I have this code snippet for the bind implementation of a Monad:
> >
> > AsyncT m >>= f = AsyncT $ \_ stp yld ->
> > let run x = (runAsyncT x) Nothing stp yld
> > yield a _ Nothing  = run $ f a
> > yield a _ (Just r) = run $ f a <> (r >>= f)
> > in m Nothing stp yield
> >
> > I want to have multiple versions of this implementation parameterized by
> a
> > function, like this:
> >
> > bindWith k (AsyncT m) f = AsyncT $ \_ stp yld ->
> > let run x = (runAsyncT x) Nothing stp yld
> > yield a _ Nothing  = run $ f a
> > yield a _ (Just r) = run $ f a `k` (bindWith k r f)
> > in m Nothing stp yield
> >
> > And then the bind function becomes:
> >
> > (>>=) = bindWith (<>)
> >
> > But this leads to a performance degradation of more than 10%. inlining
> does
> > not help, I tried INLINE pragma as well as the "inline" GHC builtin. I
> > thought this should be a more or less straightforward replacement making
> the
> > second version equivalent to the first one. But apparently there is
> > something going on here that makes it perform worse.
> >
> > I did not look at the core, stg or asm yet. Hoping someone can quickly
> > comment on it. Any ideas why is it so? Can this be worked around somehow?
> >
> > Thanks,
> > Harendra
> >
>
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs=02%7C01%7Csimonpj%40microsoft.com%7C5ff3c69fb9d447c47b5908d4f6b832de%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636404718373134824=zyHYozym6TzL61Tq5CSERjqhKlxr%2ByV0j%2FyHtxmXmVE%3D=0>
> >
>
>
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance degradation when factoring out common code

[Harendra Kumar]

I tested my code on GHC 8.2.1, there is a good news and a bad news. The
good news is that with 8.2.1 performance of my code has improved by a huge
margin of around 20%, I had reported a similar perf improvement earlier in
another one of my package with 8.2.

The bad news is that the problem that I reported today is not fixed. I am
still seeing a 15% difference between automatic and manual inlining. I
guess I should raise a ticket.

-harendra

On 8 September 2017 at 17:10, Mikolaj Konarski <mikolaj.konar...@gmail.com>
wrote:

> Hello,
>
> I've had a similar problem that's been fixed in 8.2.1:
>
> https://ghc.haskell.org/trac/ghc/ticket/12603
>
> You can also use some extreme global flags, such as
>
> ghc-options: -fexpose-all-unfoldings -fspecialise-aggressively
>
> to get most the GHC subtlety and shyness out of the way
> when experimenting.
>
> Good luck
> Mikolaj
>
>
>
> On Fri, Sep 8, 2017 at 11:21 AM, Harendra Kumar
> <harendra.ku...@gmail.com> wrote:
> > Hi,
> >
> > I have this code snippet for the bind implementation of a Monad:
> >
> > AsyncT m >>= f = AsyncT $ \_ stp yld ->
> > let run x = (runAsyncT x) Nothing stp yld
> > yield a _ Nothing  = run $ f a
> > yield a _ (Just r) = run $ f a <> (r >>= f)
> > in m Nothing stp yield
> >
> > I want to have multiple versions of this implementation parameterized by
> a
> > function, like this:
> >
> > bindWith k (AsyncT m) f = AsyncT $ \_ stp yld ->
> > let run x = (runAsyncT x) Nothing stp yld
> > yield a _ Nothing  = run $ f a
> > yield a _ (Just r) = run $ f a `k` (bindWith k r f)
> > in m Nothing stp yield
> >
> > And then the bind function becomes:
> >
> > (>>=) = bindWith (<>)
> >
> > But this leads to a performance degradation of more than 10%. inlining
> does
> > not help, I tried INLINE pragma as well as the "inline" GHC builtin. I
> > thought this should be a more or less straightforward replacement making
> the
> > second version equivalent to the first one. But apparently there is
> > something going on here that makes it perform worse.
> >
> > I did not look at the core, stg or asm yet. Hoping someone can quickly
> > comment on it. Any ideas why is it so? Can this be worked around somehow?
> >
> > Thanks,
> > Harendra
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance degradation when factoring out common code

[Harendra Kumar]

I should also point out that I saw performance improvements by manually
factoring out and propagating some common expressions to outer loops in
performance sensitive paths. Now I have made this a habit to do this
manually. Not sure if something like this has also been fixed with that
ticket or some other ticket.

-harendra

On 8 September 2017 at 17:34, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> Thanks Mikolaj! I have seen some surprising behavior quite a few times
> recently and I was wondering whether GHC should do better. In one case I
> had to use SPECIALIZE very aggressively, in another version of the same
> code it worked well without that. I have been doing a lot of trial and
> error with the INLINE/NOINLINE pragmas to figure out what the right
> combination is. Sometimes it just feels like black magic, because I cannot
> find a rationale to explain the behavior. I am not sure if there are any
> more such problems lurking in, perhaps this is an area where some
> improvement looks possible.
>
> -harendra
>
>
> On 8 September 2017 at 17:10, Mikolaj Konarski <mikolaj.konar...@gmail.com
> > wrote:
>
>> Hello,
>>
>> I've had a similar problem that's been fixed in 8.2.1:
>>
>> https://ghc.haskell.org/trac/ghc/ticket/12603
>>
>> You can also use some extreme global flags, such as
>>
>> ghc-options: -fexpose-all-unfoldings -fspecialise-aggressively
>>
>> to get most the GHC subtlety and shyness out of the way
>> when experimenting.
>>
>> Good luck
>> Mikolaj
>>
>>
>>
>> On Fri, Sep 8, 2017 at 11:21 AM, Harendra Kumar
>> <harendra.ku...@gmail.com> wrote:
>> > Hi,
>> >
>> > I have this code snippet for the bind implementation of a Monad:
>> >
>> > AsyncT m >>= f = AsyncT $ \_ stp yld ->
>> > let run x = (runAsyncT x) Nothing stp yld
>> > yield a _ Nothing  = run $ f a
>> > yield a _ (Just r) = run $ f a <> (r >>= f)
>> > in m Nothing stp yield
>> >
>> > I want to have multiple versions of this implementation parameterized
>> by a
>> > function, like this:
>> >
>> > bindWith k (AsyncT m) f = AsyncT $ \_ stp yld ->
>> > let run x = (runAsyncT x) Nothing stp yld
>> > yield a _ Nothing  = run $ f a
>> > yield a _ (Just r) = run $ f a `k` (bindWith k r f)
>> > in m Nothing stp yield
>> >
>> > And then the bind function becomes:
>> >
>> > (>>=) = bindWith (<>)
>> >
>> > But this leads to a performance degradation of more than 10%. inlining
>> does
>> > not help, I tried INLINE pragma as well as the "inline" GHC builtin. I
>> > thought this should be a more or less straightforward replacement
>> making the
>> > second version equivalent to the first one. But apparently there is
>> > something going on here that makes it perform worse.
>> >
>> > I did not look at the core, stg or asm yet. Hoping someone can quickly
>> > comment on it. Any ideas why is it so? Can this be worked around
>> somehow?
>> >
>> > Thanks,
>> > Harendra
>> >
>> > ___
>> > ghc-devs mailing list
>> > ghc-devs@haskell.org
>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> >
>>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Performance degradation when factoring out common code

[Harendra Kumar]

Thanks Mikolaj! I have seen some surprising behavior quite a few times
recently and I was wondering whether GHC should do better. In one case I
had to use SPECIALIZE very aggressively, in another version of the same
code it worked well without that. I have been doing a lot of trial and
error with the INLINE/NOINLINE pragmas to figure out what the right
combination is. Sometimes it just feels like black magic, because I cannot
find a rationale to explain the behavior. I am not sure if there are any
more such problems lurking in, perhaps this is an area where some
improvement looks possible.

-harendra


On 8 September 2017 at 17:10, Mikolaj Konarski <mikolaj.konar...@gmail.com>
wrote:

> Hello,
>
> I've had a similar problem that's been fixed in 8.2.1:
>
> https://ghc.haskell.org/trac/ghc/ticket/12603
>
> You can also use some extreme global flags, such as
>
> ghc-options: -fexpose-all-unfoldings -fspecialise-aggressively
>
> to get most the GHC subtlety and shyness out of the way
> when experimenting.
>
> Good luck
> Mikolaj
>
>
>
> On Fri, Sep 8, 2017 at 11:21 AM, Harendra Kumar
> <harendra.ku...@gmail.com> wrote:
> > Hi,
> >
> > I have this code snippet for the bind implementation of a Monad:
> >
> > AsyncT m >>= f = AsyncT $ \_ stp yld ->
> > let run x = (runAsyncT x) Nothing stp yld
> > yield a _ Nothing  = run $ f a
> > yield a _ (Just r) = run $ f a <> (r >>= f)
> > in m Nothing stp yield
> >
> > I want to have multiple versions of this implementation parameterized by
> a
> > function, like this:
> >
> > bindWith k (AsyncT m) f = AsyncT $ \_ stp yld ->
> > let run x = (runAsyncT x) Nothing stp yld
> > yield a _ Nothing  = run $ f a
> > yield a _ (Just r) = run $ f a `k` (bindWith k r f)
> > in m Nothing stp yield
> >
> > And then the bind function becomes:
> >
> > (>>=) = bindWith (<>)
> >
> > But this leads to a performance degradation of more than 10%. inlining
> does
> > not help, I tried INLINE pragma as well as the "inline" GHC builtin. I
> > thought this should be a more or less straightforward replacement making
> the
> > second version equivalent to the first one. But apparently there is
> > something going on here that makes it perform worse.
> >
> > I did not look at the core, stg or asm yet. Hoping someone can quickly
> > comment on it. Any ideas why is it so? Can this be worked around somehow?
> >
> > Thanks,
> > Harendra
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Performance degradation when factoring out common code

[Harendra Kumar]

Hi,

I have this code snippet for the bind implementation of a Monad:

AsyncT m >>= f = AsyncT $ \_ stp yld ->
let run x = (runAsyncT x) Nothing stp yld
yield a _ Nothing  = run $ f a
yield a _ (Just r) = run $ f a <> (r >>= f)
in m Nothing stp yield

I want to have multiple versions of this implementation parameterized by a
function, like this:

bindWith k (AsyncT m) f = AsyncT $ \_ stp yld ->
let run x = (runAsyncT x) Nothing stp yld
yield a _ Nothing  = run $ f a
yield a _ (Just r) = run $ f a `k` (bindWith k r f)
in m Nothing stp yield

And then the bind function becomes:

(>>=) = bindWith (<>)
But this leads to a performance degradation of more than 10%. inlining does
not help, I tried INLINE pragma as well as the "inline" GHC builtin. I
thought this should be a more or less straightforward replacement making
the second version equivalent to the first one. But apparently there is
something going on here that makes it perform worse.

I did not look at the core, stg or asm yet. Hoping someone can quickly
comment on it. Any ideas why is it so? Can this be worked around somehow?

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Where to raise an issue for the stm library

[Harendra Kumar]

Thanks Ben! You rock!

On 26 August 2017 at 09:15, Ben Gamari <b...@smart-cactus.org> wrote:

> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > Ok. So I got the bug tracker link from the hackage page which I missed
> > earlier. It will be great if the source repo page as well has a link to
> the
> > bug tracker. I have never used the bug tracker link on hackage before
> > because I usually go to the package source page and it usually has the
> > issue tracker as well (e.g. for github repos). Also, it may be good to
> have
> > it as libraries/stm instead of libraries/other in the ghc trac ticket
> page.
> >
> I've created a libraries/stm component; I'll update the stm library's
> cabal file accordingly.
>
> Cheers,
>
> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Where to raise an issue for the stm library

[Harendra Kumar]

BTW, this is the bug tracker link I got from the hackage
https://ghc.haskell.org/trac/ghc/newticket?component=libraries%20%28other%29=stm
.

On 26 August 2017 at 00:45, Harendra Kumar <harendra.ku...@gmail.com> wrote:

> Ok. So I got the bug tracker link from the hackage page which I missed
> earlier. It will be great if the source repo page as well has a link to the
> bug tracker. I have never used the bug tracker link on hackage before
> because I usually go to the package source page and it usually has the
> issue tracker as well (e.g. for github repos). Also, it may be good to have
> it as libraries/stm instead of libraries/other in the ghc trac ticket page.
>
> -harendra
>
> On 26 August 2017 at 00:36, Harendra Kumar <harendra.ku...@gmail.com>
> wrote:
>
>> Can someone tell me where to raise an issue for the stm library? The
>> hackage link sends me here http://git.haskell.org/packages/stm.git. But
>> this page provides no clue on raising issues. I wondered if GHC trac itself
>> is used for this library. I saw some other libraries in the drop down list
>> but not this one in the GHC trac ticket creation page. It is frustrating
>> that figuring out how to raise an issue is not easy for such an important
>> library.
>>
>> -harendra
>>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Where to raise an issue for the stm library

[Harendra Kumar]

Ok. So I got the bug tracker link from the hackage page which I missed
earlier. It will be great if the source repo page as well has a link to the
bug tracker. I have never used the bug tracker link on hackage before
because I usually go to the package source page and it usually has the
issue tracker as well (e.g. for github repos). Also, it may be good to have
it as libraries/stm instead of libraries/other in the ghc trac ticket page.

-harendra

On 26 August 2017 at 00:36, Harendra Kumar <harendra.ku...@gmail.com> wrote:

> Can someone tell me where to raise an issue for the stm library? The
> hackage link sends me here http://git.haskell.org/packages/stm.git. But
> this page provides no clue on raising issues. I wondered if GHC trac itself
> is used for this library. I saw some other libraries in the drop down list
> but not this one in the GHC trac ticket creation page. It is frustrating
> that figuring out how to raise an issue is not easy for such an important
> library.
>
> -harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Where to raise an issue for the stm library

[Harendra Kumar]

Can someone tell me where to raise an issue for the stm library? The
hackage link sends me here http://git.haskell.org/packages/stm.git. But
this page provides no clue on raising issues. I wondered if GHC trac itself
is used for this library. I saw some other libraries in the drop down list
but not this one in the GHC trac ticket creation page. It is frustrating
that figuring out how to raise an issue is not easy for such an important
library.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.2 generated code faster than 8.0 but slower than 7.10

[Harendra Kumar]

https://ghc.haskell.org/trac/ghc/ticket/14072

8.2.1 is (marginally) better than 7.8.4 on this benchmark, so I guess 8.4
can be better than 7.10.3.

-harendra

On 1 August 2017 at 15:42, Simon Peyton Jones <simo...@microsoft.com> wrote:

> Thanks.  Could you open a Trac ticket, and explain carefully how to
> reproduce your results?
>
>
>
> Surely 8.4 should be faster than 7.10!
>
>
>
> Simon
>
>
>
> *From:* ghc-devs [mailto:ghc-devs-boun...@haskell.org] *On Behalf Of *Harendra
> Kumar
> *Sent:* 01 August 2017 10:46
> *To:* ghc-devs@haskell.org
> *Subject:* GHC 8.2 generated code faster than 8.0 but slower than 7.10
>
>
>
> Unicode normalization library (https://github.com/harendra-
> kumar/unicode-transforms
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fharendra-kumar%2Funicode-transforms=04%7C01%7Csimonpj%40microsoft.com%7Cea675f49b17b45582c5708d4d8c6%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636371775684809111%7CUnknown%7CVW5rbm93bnx7IlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiT3RoZXIifQ%3D%3D%7C-1=4YlXVWqT4DKYHpgUcq9YF%2BiX2xH%2BLAHgI4GFCGt6zwg%3D=0>)
> shows around 10% improvement with GHC 8.2.1 when compared to GHC 8.0.1,
> across most benchmarks. However, it is still somewhat slower when compared
> to GHC 7.10.3. Here are some results:
>
>
>
> GHC 7.10.3:
>
> benchmarking unicode-transforms-text/NFD/English
> time 4.823 ms   (4.765 ms .. 4.902 ms)
>
> benchmarking unicode-transforms-text/NFD/Devanagari
> time 16.46 ms   (16.24 ms .. 16.78 ms)
>
>
> GHC 8.0.1
>
>
> benchmarking unicode-transforms-text/NFD/English
> time 6.330 ms   (6.232 ms .. 6.439 ms)
>
> benchmarking unicode-transforms-text/NFD/Devanagari
> time 18.02 ms   (17.77 ms .. 18.22 ms)
>
> GHC 8.2.1
>
>
> benchmarking unicode-transforms-text/NFD/English
> time 5.659 ms   (5.594 ms .. 5.740 ms)
>
> benchmarking unicode-transforms-text/NFD/Devanagari
> time 16.48 ms   (16.30 ms .. 16.69 ms)
>
>
>
> This library has been optimized with an objective to match or better (in
> some cases it is better) the performance of the equivalent ICU C++ library
> (compare only decompose normalization). Some of the last hurdles to match
> the best case of C++ were in the code generated by GHC. Also, for GHC 7.10
> LLVM generated code is significantly faster than GHC native, I have not yet
> tested LLVM backend with 8.2.1 so not sure if that too has improved
> correspondingly.
>
>
>
> I just wanted to share these results with the GHC devs. Thanks for all the
> good work. Also wondering which specific changes might have caused this
> improvement.
>
>
>
> Thanks,
>
> Harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC 8.2 generated code faster than 8.0 but slower than 7.10

[Harendra Kumar]

Unicode normalization library (
https://github.com/harendra-kumar/unicode-transforms) shows around 10%
improvement with GHC 8.2.1 when compared to GHC 8.0.1, across most
benchmarks. However, it is still somewhat slower when compared to GHC
7.10.3. Here are some results:

GHC 7.10.3:

benchmarking unicode-transforms-text/NFD/English
time 4.823 ms   (4.765 ms .. 4.902 ms)

benchmarking unicode-transforms-text/NFD/Devanagari
time 16.46 ms   (16.24 ms .. 16.78 ms)


GHC 8.0.1


benchmarking unicode-transforms-text/NFD/English
time 6.330 ms   (6.232 ms .. 6.439 ms)

benchmarking unicode-transforms-text/NFD/Devanagari
time 18.02 ms   (17.77 ms .. 18.22 ms)

GHC 8.2.1


benchmarking unicode-transforms-text/NFD/English
time 5.659 ms   (5.594 ms .. 5.740 ms)

benchmarking unicode-transforms-text/NFD/Devanagari
time 16.48 ms   (16.30 ms .. 16.69 ms)

This library has been optimized with an objective to match or better (in
some cases it is better) the performance of the equivalent ICU C++ library
(compare only decompose normalization). Some of the last hurdles to match
the best case of C++ were in the code generated by GHC. Also, for GHC 7.10
LLVM generated code is significantly faster than GHC native, I have not yet
tested LLVM backend with 8.2.1 so not sure if that too has improved
correspondingly.

I just wanted to share these results with the GHC devs. Thanks for all the
good work. Also wondering which specific changes might have caused this
improvement.

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Building GHC reference manual?

[Harendra Kumar]

Its well documented here:
https://ghc.haskell.org/trac/ghc/wiki/Building/Docs.

-harendra

On 11 July 2017 at 21:37, Iavor Diatchki  wrote:

> Hello,
>
> is it possible to build just the HTML for the GHC reference manual,
> without building the whole of GHC, and if so how do I do it?
>
> -Iavor
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Extensible records & mandatory/optional record fields

[Harendra Kumar]

In my application, I want some of the fields in a record to have default
values while others must always be specified by the user.  I can implement
this using the rawr records library (http://hackage.haskell.org/package/rawr)
like so:

def :: ("b" := String) -> R ("a" := Int, "b" := String)
def t = (R (#a := 0) :*: R t) -- merge with defaults

let r = def (#b := "hello")

Here field "a" has a default value (i.e. it is optional when "def" is used)
but field "b" does not have a default value and has to be explicitly
specified by the user. This is made possible by record merging feature of
rawr, making the record extensible.

I could not find an equivalent way of achieving this using overloaded
records (http://hackage.haskell.org/package/overloaded-records). Is record
merging possible or can be made possible using overloaded-records or with
the ghc overloaded records poposal? If not, is there a plan to make
something like this possible in future? Or will this never be possible with
the in built record syntax?

Note that, this is handy to simulate mandatory and optional keyword
arguments to a function by passing them in the form of a record. Something
which is available in other languages like python.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Lexical error in string continuation

[Harendra Kumar]

Hi devs,

I am making a change in runghc on the ghc master branch. When compiling the
following code (edited/new code in utils/runghc):

208 splitGhcNonGhcArgs :: [String] -> IO ([String], [String])
209 splitGhcNonGhcArgs args = do
210let (ghc, other) = break notAFlag args
211when (hasUnescapedGhcArgs ghc) $
212hPutStrLn stderr "yy\
213\ xx"

I get an error because of the string continuation at line 212. If I put the
backslashes on the same line I do not get any error. I have more string
continuations in the same file and they all work fine. This snippet works
fine with ghc-7.10.3/ghc-8.0.1 when compiled separately. Here is the error
message that I get:

utils/runghc/Main.hs:212:56: error:

lexical error in string/character literal at character 'x'

   |

212 | hPutStrLn stderr "yy\

   |^

utils/runghc/ghc.mk:30: recipe for target
'utils/runghc/dist-install/build/Main.dyn_o' failed

make[2]: *** [utils/runghc/dist-install/build/Main.dyn_o] Error 1

Makefile:122: recipe for target 'all_utils/runghc' failed

make[1]: *** [all_utils/runghc] Error 2

make[1]: Leaving directory '/vol/hosts/cueball/workspace/play/ghc'

../../mk/sub-makefile.mk:50: recipe for target 'all' failed

make: *** [all] Error 2


Any help will be appreciated. I can send the modified file if anyone wants
to reproduce/debug.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Competing with C in a simple loop

[Harendra Kumar]

On 23 December 2016 at 03:23, Christopher Done  wrote:
>
> But if you scroll down the README to the 182kb file example, you see that
> hexml takes 33us and xeno takes 111us. That's surprising to me because I'm
> doing just a walk across a string and hexml is doing a full parse. It's
> written in C, but still, 3x faster AND doing allocations and more work.
>

hexml being a full parser might fail, on the other hand your program
unconditionally walks the bytestring. Are you sure hexml is actually
completing and not aborting or short-circuiting because of a parse error or
some other error? In all other data points xeno is taking much less time
than hexml except this one. So I am suspecting it could be a problem with
the input, making hexml fail silently. I see that the file used on this
data point has japanese characters, maybe hexml is not able to handle those?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: data families syntax

[Harendra Kumar]

Answering my own question, which means I did not do enough research before
asking. I found two points that I missed earlier:

1) data families are open in contrast to value level functions i.e. you can
add more instances later on hence the family and instance keywords make
sense.

2) the syntax 'data family List :: * -> *' is actually already supported as
an alternative.

-harendra


On 13 November 2016 at 16:10, Harendra Kumar <harendra.ku...@gmail.com>
wrote:

> I am curious about why the data families syntax was chosen to be the way
> it is. For example a list can be defined as follows:
>
> data family List a
>
> data instance List Char = Empty | Cons Char (List Char)
>
> data instance List ()   = Count Int
>
>
> Instead why not define it as:
>
> data List :: * -> *
>
> data List Char = Empty | Cons Char (List Char)
>
> data List ()   = Count Int
>
>
> The latter form looks more intuitive and easy to remember to me since it
> is very similar to the way value level functions are defined (signature &
> pattern matching defs). Here we are just defining a type level function
> instead.
>
> -harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

data families syntax

[Harendra Kumar]

I am curious about why the data families syntax was chosen to be the way it
is. For example a list can be defined as follows:

data family List a

data instance List Char = Empty | Cons Char (List Char)

data instance List ()   = Count Int


Instead why not define it as:

data List :: * -> *

data List Char = Empty | Cons Char (List Char)

data List ()   = Count Int


The latter form looks more intuitive and easy to remember to me since it is
very similar to the way value level functions are defined (signature &
pattern matching defs). Here we are just defining a type level function
instead.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Improving GHC GC for latency-sensitive networked services

[Harendra Kumar]

It will be awesome if we can spread the GC work instead of stopping the
world for too long. I am a new entrant to the Haskell world but something
similar to this was the first real problem (other than lazy IO) that I
faced with GHC. While I was debugging I had to learn how the GC works to
really understand what's going on. Then I learnt to always strive to keep
the retained heap to the minimum possible. But sometimes the minimum
possible could be a lot. This blog article was sort of a deja vu for me. It
seems this is not a rare problem.

I guess, the compact regions technique as suggested by Ben can be used to
workaround the problem but it sounds like it is application aware and users
will have to discover the possibility of that solution, I might be mistaken
though. If we want GHC to work smoothly for performance critical
applications then we should perhaps find a cost effective way to solve this
in an application transparent manner.

-harendra

On 18 October 2016 at 18:33, Simon Peyton Jones via ghc-devs <
ghc-devs@haskell.org> wrote:

> |  I understand Marlow's thread-local heaps experiment circa 7.2/7.4 was
> |  abandoned because it penalized performance too much. Does the
> |  impression that there isn't the labor to maintain two GCs still hold?
> |  It seems like thread-local heaps would be pervasive.
>
> I was optimistic about thread-local heaps, but while perf did improve a
> bit, the complexity of the implementation was extremely daunting.   So we
> decided that the pain didn't justify the gain.
>
> I'm not sure it'd help much here, since the data is long-lived and might
> migrate into the global heap anyway.
>
> Most GCs rely on traversing live data. Here the live data is big. So
> really the only solution is to traverse it incrementally.  You can still
> stop-the-world, but you have to be able to resume normal execution before
> GC is complete, thus smearing GC out into a series of slices, interleaved
> with (but not necessarily in parallel with) the main application.
>
> I believe the that the OCaml runtime now has such a GC.  It'd be lovely to
> have one for GHC.
>
> But I defer to Simon M
>
> Simon
>
> |  -Original Message-
> |  From: ghc-devs [mailto:ghc-devs-boun...@haskell.org] On Behalf Of
> |  Christopher Allen
> |  Sent: 17 October 2016 18:08
> |  To: ghc-devs@haskell.org
> |  Subject: Improving GHC GC for latency-sensitive networked services
> |
> |  It'd be unfortunate if more companies trying out Haskell came to the
> |  same result:
> |  https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.
> |  pusher.com%2Flatency-working-set-ghc-gc-pick-two%2F%23comment-
> |  2866985345=01%7C01%7Csimonpj%40microsoft.com%7C04c1bc69e00c47d382
> |  2908d3f6b028d0%7C72f988bf86f141af91ab2d7cd011db47%7C1=dE1VP0u3kQ
> |  L9R7CaGTAOGswRY6SyKH72c0xG%2FOggEK0%3D=0
> |  (They gave up and rewrote the service in Golang)
> |
> |  Most of the state of the art I'm aware of (such as from Azul Systems)
> |  is from when I was using a JVM language, which isn't necessarily
> |  applicable for GHC.
> |
> |  I understand Marlow's thread-local heaps experiment circa 7.2/7.4 was
> |  abandoned because it penalized performance too much. Does the
> |  impression that there isn't the labor to maintain two GCs still hold?
> |  It seems like thread-local heaps would be pervasive.
> |
> |  Does anyone know what could be done in GHC itself to improve this
> |  situation? Stop-the-world is pretty painful when the otherwise
> |  excellent concurrency primitives are much of why you're using Haskell.
> |
> |  --- Chris Allen
> |  ___
> |  ghc-devs mailing list
> |  ghc-devs@haskell.org
> |  https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h
> |  askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
> |  devs=01%7C01%7Csimonpj%40microsoft.com%7C04c1bc69e00c47d3822908d3
> |  f6b028d0%7C72f988bf86f141af91ab2d7cd011db47%7C1=XwvaAPx%2BGqugD4
> |  Kx%2FkXiYticgBCUMkboqH9QE315EhQ%3D=0
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Notes from Ben's "contribute to ghc" discussion

[Harendra Kumar]

On 25 September 2016 at 12:48, Joachim Breitner 
wrote:

> Hi,
>
> > It will be great to have something like that. Something that you
> > figure out digging at ghc trac wiki pages, mailing lists, google
> > search etc will be a few minutes job for a mentor. It may be a bit
> > taxing on the mentors but they can limit how many newbies they are
> > mentoring and also breed new mentors to keep the cycle going.
>
> I hope and assume that already now that every possible contributor who
> has questions like this and asks (e.g. on irc) will get a helpful
> answer. Is that insufficient?


Maybe. Though irc seems to be quite popular among Haskell community and
other open source communities I have never been able to utilize it
somehow.  I don't know if there is something wrong with it or with me. I
installed an irc client logged into it once or twice but never got hooked
to it. Such questions on ghc-devs maybe a nuisance for a lot of other
people or at least that's what I felt as a newbie. I usually tend to do a
lot of homework before sending a question to ghc-devs. Maybe a ghc-newbies
like mailing list (one more list!) can give the impression of a lower
barrier for sending stupid or operational questions.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Notes from Ben's "contribute to ghc" discussion

[Harendra Kumar]

It will be great to have something like that. Something that you figure out
digging at ghc trac wiki pages, mailing lists, google search etc will be a
few minutes job for a mentor. It may be a bit taxing on the mentors but
they can limit how many newbies they are mentoring and also breed new
mentors to keep the cycle going.

-harendra

On 25 September 2016 at 11:46, Jason Dagit  wrote:

>
>
> On Sat, Sep 24, 2016 at 6:29 PM, Christopher Allen 
> wrote:
>
>> I'd be willing to help with work required to open up GHC development
>> more along multiple lines including:
>>
>> Bots/automation for Github
>>
>> Talking to Rust devs about what works, what doesn't
>>
>
> Last year I approached some folks in the rust community because I wanted
> to learn how to contribute to the rust compiler. In my experience, the
> really special thing they had was an identified pool of contributors who
> were willing and able to provide mentoring. I got hooked up with a mentor
> and that made all the difference. I had a real live person I could talk to
> about the process, the process-meta, and that person had context with me.
> Pretty much everything else was just details.
>
> GHC dev probably has mentors too, but I don't know because I've never
> thought to check or ask.
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Errors when building users guide

[Harendra Kumar]

I got to the bottom of the problem after some debugging. The short summary
is - I was using "devel2" flavor which disables doc targets. The problem
was it only disables them partially not fully. So even though I was able to
invoke the targets I got these errors and the guide is mostly built except
for the flags refs.

The long story:

devel2.mk sets: BUILD_SPHINX_HTML  = NO

But we 'include rules/sphinx.mk' in ghc.mk unconditionally. Now, since
BUILD_SPHINX_HTML is not set we filter out mkUserGuidePart from build
targets:

BUILD_DIRS := $(filter-out utils/mkUserGuidePart,$(BUILD_DIRS))

This results in an inconsistent state of the doc targets being available
but not the mkUserGuidePart rules. So even though I was able build the docs
partially I was getting these errors.

-harendra


On 23 August 2016 at 22:31, Harendra Kumar <harendra.ku...@gmail.com> wrote:
>
> Well, I tried 'make' and 'make all' (I suppose they are both same) but
that did not make any difference.
>
> -harendra
>
> On 23 August 2016 at 21:37, Ben Gamari <b...@smart-cactus.org> wrote:
>>
>> Harendra Kumar <harendra.ku...@gmail.com> writes:
>>
>> > I see errors like this when using 'make html' in the topdir of the ghc
repo:
>> >
>> Hmm, it looks like there is a missing dependency in the build system.
>> The users guide depends upon a number of ReST source files generated by
>> utils/mkUserGuidePart. I'd guess that you haven't built these. I'll look
>> in to why this is but in the meantime the workaround is just to `make
>> all` before attempting to `make html`. Sorry for the inconvenience!
>>
>> Cheers,
>>
>> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Errors when building users guide

[Harendra Kumar]

Well, I tried 'make' and 'make all' (I suppose they are both same) but that
did not make any difference.

-harendra

On 23 August 2016 at 21:37, Ben Gamari <b...@smart-cactus.org> wrote:

> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > I see errors like this when using 'make html' in the topdir of the ghc
> repo:
> >
> Hmm, it looks like there is a missing dependency in the build system.
> The users guide depends upon a number of ReST source files generated by
> utils/mkUserGuidePart. I'd guess that you haven't built these. I'll look
> in to why this is but in the meantime the workaround is just to `make
> all` before attempting to `make html`. Sorry for the inconvenience!
>
> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Errors when building users guide

[Harendra Kumar]

I see errors like this when using 'make html' in the topdir of the ghc repo:




/vol/hosts/cueball/workspace/github/ghc/docs/users_guide/flags.rst:15:
SEVERE: Problems with "include" directive path:

InputError: [Errno 2] No such file or directory:
'docs/users_guide/flags-verbosity.gen.rst'.



What am I doing wrong or not doing?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Style or bug in user manual?

[Harendra Kumar]

Thanks! I fixed it.

On 22 August 2016 at 21:29, Ben Gamari <b...@smart-cactus.org> wrote:

> Harendra Kumar <harendra.ku...@gmail.com> writes:
>
> > I was working to fix some documentation and noticed that the user manual
> > lists options like this:
> >
> > -package⟨pkg⟩
> >
> > Notice that there is no space between the argument and the option. Should
> > this be fixed? All options on this page seem to be similarly formatted
> > which makes me wonder whether its by design?
> >
> Looking at docs/users_guide/packages.rst it appears there are indeed
> spaces in the source, which suggests that this is a bug in the parser in
> docs/users_guide/conf.py. Perhaps you could open a ticket (and of course
> feel free to take a stab at a solution)?
>
> Cheers,
>
> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Style or bug in user manual?

[Harendra Kumar]

I was working to fix some documentation and noticed that the user manual
lists options like this:

-package⟨pkg⟩

Notice that there is no space between the argument and the option. Should
this be fixed? All options on this page seem to be similarly formatted
which makes me wonder whether its by design?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Using stringize and string concatenation in ghc preprocessing

[Harendra Kumar]

Hi,

To reduce boilerplate code in an FFI implementation file I am trying to use
the stringizing and string concatenation features of the C
preprocessor. Since ghc passes '-traditional' to the preprocessor which
disables these features I thought I can pass my own flags to the
preprocessor like this:

{-# OPTIONS_GHC -optP -E -optP -undef #-}

But "-optP" seems to only append to the flags that GHC already passes and
gcc has no "-no-traditional" option to undo the effect of the
"-traditional" that GHC has already passed. I think "-optP" should override
the flags passed by ghc rather than appending to them. Is there a reason
not to do that?

Is there any other better way to achieve this? What is the standard way of
doing this if any?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: ghc command line arguments parsing

[Harendra Kumar]

On 19 August 2016 at 14:42, Sven Panne  wrote:

>
> Hmmm, do we need '--ghc-args=' at all when we have '--' as the border
> between the 2 "argument worlds"? Without thinking too much about it, :-} I
> guess we don't. This would be consistent with how e.g. "stack exec" or
> "gdb" work. An explicit option --pass-this-to-some-progXY is only needed
> when there are more than 2 "argument worlds" involved, otherwise '--' is
> enough, and also easier to type/remember.
>

Hmm, I have been there :-) There is a slight gotcha here. This is how the
runghc command looks like:

runghc [runghc flags] [--] [GHC flags] progname [prog args]

The regular '--' separation works well when we have only two worlds; we
consume upto '--' and give the rest opaquely to the other one. Here we have
three  scopes, we need to separate runghc flags, GHC flags, program and its
args. In the above command we cannot distinguish a GHC arg from progname.

One way to solve that is to put another mandatory '--' after the GHC flags
so that GHC flags are always delimited. With that here are the valid forms
of the command:

1) runghc [runghc flags] progname [prog args]

2) runghc [runghc flags] -- progname [prog args]
3) runghc [runghc flags] -- [GHC flags] -- progname [prog args]

For simplicity we will only pass GHC args using the 3rd form of the command
even if they are not conflicting with runghc flags. Note that these changes
will be incompatible with the previous implementation. Does this sound ok?
Or we keep the good old '--ghc-arg='?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: ghc command line arguments parsing

[Harendra Kumar]

On 19 August 2016 at 12:31, Sven Panne <svenpa...@gmail.com> wrote:

> 2016-08-18 19:59 GMT+02:00 Harendra Kumar <harendra.ku...@gmail.com>:
>
>> [...]  It only parses a flag which takes an argument. [...]
>>
>
> o_O AFAICT, this is even more non-standard and quite surprising...
>
>
>> As I explained above, I would prefer to keep this bug :-) and document it
>> especially for runghc as a better alternative to --ghc-arg=foo . [...]
>>
>
> While I partly understand your motivation, convenience is often the worst
> reason for doing something. Consistency is IMHO infinitely more valuable
> than having various inconsistent ad-hoc "convenient" shortcuts here and
> there. If you look around the tool landscape, commandline argument parsing
> is already less consistent than it should be (try e.g.
> ps/tar/unzip/df/java/git/...), and I don't think we should make this
> situation worse by adding yet another variation.
>
> As long as we have a way to pass arguments down the pipeline (as we
> obviously have), thing are fine IMHO. This situation is not much different
> from the common -Wl,foo option for GCC and friends to pass arguments to the
> linker. And to be honest: How often do you really *type* long commandlines
> compared to putting them into some kind of script?
>

Funnily consistency was my point too and not convenience. I totally agree
that consistency is more important. The end objective is less referring to
manuals and rely more on intuition and consistency. Its all about making it
easier to remember and not about typing less or more. But what I find
consistent you find it inconsistent :-) Maybe I am squinting too much to
see it that way. Let me explain it a bit.

I see each flag as a single unit including its argument. In fact command
line tools often combine the flag and value e.g. -f/path/to/ghc and -f
/path/to/ghc mean the same thing to runghc. This is usually done for a
single character flag only because it is unambiguous to parse even with
using a space separator. If we generalize this we can also allow a space in
the combined value if escaped e.g. -package foo and "-package foo" can be
considered the same thing. If we look at flags in this way they in fact
look more consistent - everything i.e. all flags start with a dash.

In fact I discovered that it works this way because I intuitively guessed
it, runghc CLI help did not have any mention of how to pass such args to
ghc. I discovered '--ghc-arg=' only later in users guide.

I would be happier with a gcc style -Wl,foo or -Xlinker way because it is
more consistent compared to how runghc passes options to ghc. runghc has
two ways of handling ghc options. Options starting with a dash are passed
even without having to use '--ghc-arg=' and if they conflict with runghc
options then we use '--' to separate the two. If a ghc option does not
start with a dash then we use '--ghc-arg'. Three different ways! too
complicated to remember or think about. As you said I would be happier with
an inconvenient to type but consistent alternative where we use
'--ghc-arg=' for everything and get rid of the other ways. Originally I was
trying to do it the other way round by getting rid of '--ghc-arg' instead.
But this perhaps sounds easier to think about. Let me know what you think.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: ghc command line arguments parsing

[Harendra Kumar]

On 18 August 2016 at 20:51, Sven Panne <svenpa...@gmail.com> wrote:

> 2016-08-17 16:37 GMT+02:00 Harendra Kumar <harendra.ku...@gmail.com>:
>
>> ghc accepts a flag and its argument as a single quoted or escaped
>> argument as well. For example all of the following are equivalent:
>>
>> ghc -package foo
>> ghc "-package foo"
>> ghc -package\ foo
>>
>> Is this by design or accidental?
>>
>
> I would call this a bug, probably some missing quoting somewhere: Most
> commands I know don't do splitting on the arguments for themselves. Just
> try e.g.
>
>gcc "-v -v"
>

Initially I too thought the same but then I did some more testing and it
turned out that the way ghc does not really parse it exactly in the same
way as independent unquoted arguments. It only parses a flag which takes an
argument. For example:
ghc "-package foo" -- parses foo as an argument to the -package flag
ghc "-package foo -v" -- is not the same as "-package foo" and "-v"
arguments passed independently, it is "foo -v" as the argument to "-package"

So it essentially allows packing a flag and its argument together. Which
provides a nice way to pass flags and their values as a single unit. This
may be accidental to begin with but I liked this just because it makes
passing opaque flags combined with arguments from a wrapper program like
runghc much more convenient. I cannot think of any downside of this
behavior unless someone points out otherwise.


>
>> This has a nice side effect to make passing ghc arguments via rughc
>> simple. For example runghc "-package foo" or runghc -package\ foo will pass
>> "-package foo" to ghc. The alternative and documented way to achieve the
>> same effect is runghc -package --ghc-arg=foo which is not that convenient.
>>
>> My question is - can we rely on this way of parsing? If it is not by
>> design, does it make sense to legalize and therefore document this?
>>
>
> And my question is: Can we fix this bug? :-D If this is not considered a
> bug, we should be very explicit about this deviation from standard behavior
> in the documentation, including the reasons for it.
>

As I explained above, I would prefer to keep this bug :-) and document it
especially for runghc as a better alternative to --ghc-arg=foo . So instead
of saying:

runghc -package --ghc-arg=text -package --ghc-arg=turtle hello.hs
We can just say:

runghc "-package text" "-package turtle" hello.hs

 -harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

ghc command line arguments parsing

[Harendra Kumar]

Hi,

ghc accepts a flag and its argument as a single quoted or escaped argument
as well. For example all of the following are equivalent:

ghc -package foo
ghc "-package foo"
ghc -package\ foo

Is this by design or accidental?

This has a nice side effect to make passing ghc arguments via rughc simple.
For example runghc "-package foo" or runghc -package\ foo will pass
"-package foo" to ghc. The alternative and documented way to achieve the
same effect is runghc -package --ghc-arg=foo which is not that convenient.

My question is - can we rely on this way of parsing? If it is not by
design, does it make sense to legalize and therefore document this?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Behavior of GHC_PACKAGE_PATH

[Harendra Kumar]

Hi,

As per the GHC manual (https://downloads.haskell.
org/~ghc/master/users-guide/packages.html#the-ghc-package-
path-environment-variable), packages which come earlier in the
GHC_PACKAGE_PATH supersede the ones which come later. But that does not
seem to be the case always.

I am dealing with a case where I have multiple versions of a package in
different databases. I am passing the list of package dbs via
GHC_PACKAGE_PATH and ghc is picking the one which comes later. I expect it
to pick the one which comes earlier in the path.

In one case the package being picked from a package db which comes later is
a newer version which makes me wonder if GHC always prefers a newer
version. In another case both the versions are same but still GHC is using
the one which come later in the package path. Are there any undocumented
factors into play here?

None of the packages is broken (as reported by ghc-pkg check).

See https://github.com/commercialhaskell/stack/issues/1957#issuecomment-
239655912 for more details on where it is coming from.

Thanks,
Harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: CMM-to-ASM: Register allocation wierdness

[Harendra Kumar]

On 19 June 2016 at 14:03, Ben Gamari <b...@smart-cactus.org> wrote:

>
> Indeed it would be great if you could provide the program that produced
> this code.
>
> >> It would be great to open Trac tickets to track some of the optimization


Ok, I created an account on ghc trac and raised two tickets: #12231 &
#12232. Yay!  I also added the code branch to reproduce this on github (
https://github.com/harendra-kumar/unicode-transforms/tree/ghc-trac-12231).

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Force GC calls out of the straight line execution path

[Harendra Kumar]

On 17 June 2016 at 17:30, Ben Gamari  wrote:
>
>
> [1] https://phabricator.haskell.org/D2339


Awesome!

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: CMM-to-ASM: Register allocation wierdness

[Harendra Kumar]

Thanks Ben! I have my responses inline below.

On 16 June 2016 at 18:07, Ben Gamari <b...@smart-cactus.org> wrote:

>
> Indeed; I've opened D2335 [1] to reenable -fregs-graph and add an
> appropriate note to the users guide.
>

Thanks! That was quick.


> For the record, I have also struggled with register spilling issues in
> the past. See, for instance, #10012, which describes a behavior which
> arises from the C-- sinking pass's unwillingness to duplicate code
> across branches. While in general it's good to avoid the code bloat that
> this duplication implies, in the case shown in that ticket duplicating
> the computation would be significantly less code than the bloat from
> spilling the needed results.
>

Not sure if this is possible but when unsure we can try both and compare if
the duplication results in significantly more code than no duplication and
make a decision based on that. Though that will slow down the compilation.
Maybe we can bundle slower passes in something like -O3, meaning it will be
slow and may or may not provide better results?


> > But I found a few interesting optimizations that llvm did. For example,
> > there was a heap adjustment and check in the looping path which was
> > redundant and was readjusted in the loop itself without use. LLVM either
> > removed the redundant  _adjustments_ in the loop or moved them out of the
> > loop. But it did not remove the corresponding heap _checks_. That makes
> me
> > wonder if the redundant heap checks can also be moved or removed. If we
> can
> > do some sort of loop analysis at the CMM level itself and avoid or remove
> > the redundant heap adjustments as well as checks or at least float them
> out
> > of the cycle wherever possible. That sort of optimization can make a
> > significant difference to my case at least. Since we are explicitly aware
> > of the heap at the CMM level there may be an opportunity to do better
> than
> > llvm if we optimize the generated CMM or the generation of CMM itself.
> >
> Very interesting, thanks for writing this down! Indeed if these checks
> really are redundant then we should try to avoid them. Do you have any
> code you could share that demosntrates this?
>

The gist that I provided in this email thread earlier demonstrates it. Here
it is again:

https://gist.github.com/harendra-kumar/7d34c6745f604a15a872768e57cd2447

If you look at the CMM trace in the gist. Start at label c4ic where we
allocate space on heap (+48). Now, there are many possible paths from this
point on some of those use the heap and some don't. I have marked those
which use the heap by curly braces, the rest do not use it at all.
1) c4ic (allocate) -> c4mw -> {c4pv} -> ...
2) c4ic (allocate) -> c4mw -> c4pw -> ((c4pr -> ({c4pe} -> ... | c4ph ->
...)) | cp4ps -> ...)

If we can place this allocation at both c4pv and c4pe instead of the common
parent then we can save the fast path from this check.  The same thing
applies to the allocation at label c4jd as well.

I have the code to produce this CMM, I can commit it on a branch and leave
it in the github repository so that we can use it for fixing.


> It would be great to open Trac tickets to track some of the optimization
>

Will do.


> There is indeed a question of where we wish to focus our optimization
> efforts. However, I think using LLVM exclusively would be a mistake.
> LLVM is a rather large dependency that has in the past been rather
> difficult to track (this is why we now only target one LLVM release in a
> given GHC release). Moreover, it's significantly slower than our
> existing native code generator. There are a number of reasons for this,
> some of which are fixable. For instance, we currently make no effort to
> tell
> LLVM which passes are worth running and which we've handled; this is
> something which should be fixed but will require a rather significant
> investment by someone to determine how GHC's and LLVM's passes overlap,
> how they interact, and generally which are helpful (see GHC #11295).
>
> Furthermore, there are a few annoying impedance mismatches between Cmm
> and LLVM's representation. This can be seen in our treatment of proc
> points: when we need to take the address of a block within a function
> LLVM requires that we break the block into a separate procedure, hiding
> many potential optimizations from the optimizer. This was discussed
> further on this list earlier this year [2]. It would be great to
> eliminate proc-point splitting but doing so will almost certainly
> require cooperation from LLVM.
>

It sounds like we need to continue with both for now and see how the llvm
option pans out. There is clearly no reason for a decisive tilt towards
llvm in near future.

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: CMM-to-ASM: Register allocation wierdness

[Harendra Kumar]

That's a nice read, thanks for the pointer. I agree with the solution
presented there. If we can do that it will be awesome. If help is needed I
can spend some time on it.

One of the things that I noticed is that the code can be optimized
significantly if we know the common case so that we can optimize that path
at the expense of less common path. At times I saw wild difference in
performance just by a very small change in the source. I could attribute
the difference to code blocks having moved and differently placed jump
instructions or change in register allocations impacting the common case
more. This could be avoided if we know the common case.

The common case is not visible or obvious to low level tools. It is easier
to write the code in a low level language like C such that it is closer to
how it will run on the processor, we can also easily influence gcc from the
source level. It is harder to do the same in a high level language like
Haskell. Perhaps there is no point in doing so. What we can do instead is
to use the llvm toolchain to perform feedback directed optimization and it
will adjust the low level code accordingly based on your feedback runs.
That will be entirely free since it can be done at the llvm level.

My point is that it will pay off in things like that if we invest in
integrating llvm better.

-harendra

On 16 June 2016 at 16:48, Karel Gardas <karel.gar...@centrum.cz> wrote:

> On 06/16/16 12:53 PM, Harendra Kumar wrote:
>
>> A thought that came to my mind was whether we should focus on getting
>> better code out of the llvm backend or the native code generator. LLVM
>> seems pretty good at the specialized task of code generation and low
>> level optimization, it is well funded, widely used and has a big
>> community support. That allows us to leverage that huge effort and take
>> advantage of the new developments. Does it make sense to outsource the
>> code generation and low level optimization tasks to llvm and ghc
>> focussing on higher level optimizations which are harder to do at the
>> llvm level? What are the downsides of using llvm exclusively in future?
>>
>
> Good reading IMHO about the topic is here:
> https://ghc.haskell.org/trac/ghc/wiki/ImprovedLLVMBackend
>
> Cheers,
> Karel
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: CMM-to-ASM: Register allocation wierdness

[Harendra Kumar]

On 16 June 2016 at 13:59, Ben Gamari  wrote:
>
> It actually came to my attention while researching this that the
> -fregs-graph flag is currently silently ignored [2]. Unfortunately this
> means you'll need to build a new compiler if you want to try using it.

Yes I did try -fregs-graph and -fregs-iterative both. To debug why nothing
changed I had to compare the executables produced with and without the
flags and found them identical.  A note in the manual could have saved me
some time since that's the first place to go for help. I was wondering if I
am making a mistake in the build and if it is not being rebuilt
properly. Your note confirms my observation, it indeed does not change
anything.

> All-in-all, the graph coloring allocator is in great need of some love;
> Harendra, perhaps you'd like to have a try at dusting it off and perhaps
> look into why it regresses in compiler performance? It would be great if
> we could use it by default.

Yes, I can try that. In fact I was going in that direction and then stopped
to look at what llvm does. llvm gave me impressive results in some cases
though not so great in others. I compared the code generated by llvm and it
perhaps did a better job in theory (used fewer instructions) but due to
more spilling the end result was pretty similar.

But I found a few interesting optimizations that llvm did. For example,
there was a heap adjustment and check in the looping path which was
redundant and was readjusted in the loop itself without use. LLVM either
removed the redundant  _adjustments_ in the loop or moved them out of the
loop. But it did not remove the corresponding heap _checks_. That makes me
wonder if the redundant heap checks can also be moved or removed. If we can
do some sort of loop analysis at the CMM level itself and avoid or remove
the redundant heap adjustments as well as checks or at least float them out
of the cycle wherever possible. That sort of optimization can make a
significant difference to my case at least. Since we are explicitly aware
of the heap at the CMM level there may be an opportunity to do better than
llvm if we optimize the generated CMM or the generation of CMM itself.

A thought that came to my mind was whether we should focus on getting
better code out of the llvm backend or the native code generator. LLVM
seems pretty good at the specialized task of code generation and low level
optimization, it is well funded, widely used and has a big community
support. That allows us to leverage that huge effort and take advantage of
the new developments. Does it make sense to outsource the code generation
and low level optimization tasks to llvm and ghc focussing on higher level
optimizations which are harder to do at the llvm level? What are the
downsides of using llvm exclusively in future?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Force GC calls out of the straight line execution path

[Harendra Kumar]

Hi Ömer,

I just checked and I do not see this behavior in 8.0.1 native code, it is
only seen in 7.10.3. So I guess this got fixed in 8.0.

How about the llvm generated code? Can we control this behavior for llvm
generated code as well? I first noticed this in llvm generated code with
ghc-7.10.3. I do not yet have llvm-3.7 installed to verify the same on
8.0.1.

-harendra

On 13 June 2016 at 18:04, Ömer Sinan Ağacan <omeraga...@gmail.com> wrote:

> Hi Harendra,
>
> Would it be possible for you to provide a minimal example that compiles to
> such
> assembly? It's hard to tell if this is an easy case.
>
> Also, just to make sure, you're using -O, right? (I'm not sure if we have a
> related transformation enabled with -O but just to make sure...)
>
> 2016-06-13 7:23 GMT-04:00 Harendra Kumar <harendra.ku...@gmail.com>:
> > Hi,
> >
> > I noticed in the generated code (llvm as well as native) that in some
> cases
> > the GC calls are in the straight path and the regular code is out of the
> > straight line path. Like this:
> >
> > => 0x408fc0:  lea0x30(%r12),%rax
> > => 0x408fc5:  cmp0x358(%r13),%rax
> > => 0x408fcc:  jbe0x408fe9   # notice jbe instead of
> ja
> > i.e. branch taken in normal case
> >
> > I tried to count in how many cases its happening in my executable and
> found
> > that its only a small percentage (4-6%) of cases but those cases include
> the
> > code which runs 99% of the time in my benchmark. Though it does not make
> a
> > whole lot of difference but the difference is perceptible and especially
> > when it is in a tight loop.
> >
> > Is it possible to somehow force all the GC calls out of the line during
> code
> > generation? Has it been thought/discussed before?
> >
> > -harendra
> >
> >
> >
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Force GC calls out of the straight line execution path

[Harendra Kumar]

Hi,

I noticed in the generated code (llvm as well as native) that in some cases
the GC calls are in the straight path and the regular code is out of the
straight line path. Like this:

=> 0x408fc0:  lea0x30(%r12),%rax
=> 0x408fc5:  cmp0x358(%r13),%rax
=> 0x408fcc:  jbe0x408fe9   # notice jbe instead of ja
i.e. branch taken in normal case

I tried to count in how many cases its happening in my executable and found
that its only a small percentage (4-6%) of cases but those cases include
the code which runs 99% of the time in my benchmark. Though it does not
make a whole lot of difference but the difference is perceptible and
especially when it is in a tight loop.

Is it possible to somehow force all the GC calls out of the line during
code generation? Has it been thought/discussed before?

-harendra
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: CMM-to-ASM: Register allocation wierdness

[Harendra Kumar]

My earlier experiment was on GHC-7.10.3. I repeated this on GHC-8.0.1 and
the assembly traced was exactly the same except for a marginal improvement.
The 8.0.1 code generator removed the r14/r11 swap but the rest of the
register ring shift remains the same. I have updated the github gist with
the 8.0.1 trace:

https://gist.github.com/harendra-kumar/7d34c6745f604a15a872768e57cd2447

thanks,
harendra

On 13 June 2016 at 00:08, Harendra Kumar <harendra.ku...@gmail.com> wrote:

> Hi,
>
> I am implementing unicode normalization in Haskell. I challenged myself to
> match the performance with the best C/C++ implementation, the best being
> the ICU library. I am almost there, beating it in one of the benchmarks and
> within 30% for others. I am out of all application level tricks that I
> could think of and now need help from the compiler.
>
> I started with a bare minimum loop and adding functionality incrementally
> watching where the performance trips. At one point I saw that adding just
> one 'if' condition reduced the performance by half. I looked at what's
> going on at the assembly level. Here is a github gist of the assembly
> instructions executed in the fast path of the loop, corresponding cmm
> snippets and also the full cmm corresponding to the loop:
>
> https://gist.github.com/harendra-kumar/7d34c6745f604a15a872768e57cd2447
>
> I have annotated the assembly code with labels matching the corresponding
> CMM.
>
> With the addition of another "if" condition the loop which was pretty
> simple till now suddenly got bloated with a lot of register reassignments.
> Here is a snippet of the register movements added:
>
> # _n4se:
> # swap r14 <-> r11
> => 0x408d6b: mov%r11,0x98(%rsp)
> => 0x408d73: mov%r14,%r11
> => 0x408d76: mov0x98(%rsp),%r14
>
> # reassignments
> # rbx -> r10 -> r9 -> r8 -> rdi -> rsi -> rdx -> rcx -> rbx
> => 0x408d7e: mov%rbx,0x90(%rsp)
> => 0x408d86: mov%rcx,%rbx
> => 0x408d89: mov%rdx,%rcx
> => 0x408d8c: mov%rsi,%rdx
> => 0x408d8f: mov%rdi,%rsi
> => 0x408d92: mov%r8,%rdi
> => 0x408d95: mov%r9,%r8
> => 0x408d98: mov%r10,%r9
> => 0x408d9b: mov0x90(%rsp),%r10
> .
> .
> .
> loop logic here which uses only %rax, %r10 and %r9 .
> .
> .
> .
> # _n4s8:
> # shuffle back to original assignments
> => 0x4090dc: mov%r14,%r11
> => 0x4090df: mov%r9,%r10
> => 0x4090e2: mov%r8,%r9
> => 0x4090e5: mov%rdi,%r8
> => 0x4090e8: mov%rsi,%rdi
> => 0x4090eb: mov%rdx,%rsi
> => 0x4090ee: mov%rcx,%rdx
> => 0x4090f1: mov%rbx,%rcx
> => 0x4090f4: mov%rax,%rbx
> => 0x4090f7: mov0x88(%rsp),%rax
>
> => 0x4090ff: jmpq   0x408d2a
>
>
> The registers seem to be getting reassigned here, data flowing from one to
> the next. In this particular path a lot of these register movements seem
> unnecessary and are only undone at the end without being used.
>
> Maybe this is because these are reusable blocks and the movement is
> necessary when used in some other path?
>
> Can this be avoided? Or at least avoided in a certain fast path somehow by
> hinting the compiler? Any pointers to the GHC code will be appreciated. I
> am not yet much familiar with the GHC code but can dig deeper pretty
> quickly. But before that I hope someone knowledgeable in this area can shed
> some light on this at a conceptual level or if at all it can be improved. I
> can provide more details and experiment more if needed.
>
> Thanks,
> Harendra
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs