profiling a constructor?
Hi devs, Is there a way to count the number of times a particular constructor is allocated? I might want to know, say, the total number of cons cells allocated over the course of a program, or (in my actual case) the total number of FunTy cells allocated over the course of a program. I can try to do this with an SCC. But it's very clunky: * The constructor may be used in a number of places; each such place would need the SCC. * SCCs can interfere with optimizations. In my use case, this would negate the usefulness of the exercise entirely, as I think some of the structures I wish to observe should never come into being, due to optimizations (e.g. case-of-known-constructor after inlining). Surely, the RTS must be able to count the number of times a constructor is used. But is there any way to access such a feature? If others agree that this would sometimes be helpful, perhaps we can build the feature. Now is not the first time I've wanted this. Thanks! Richard ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: profiling a constructor?
Yes, you can use +RTS -hT, which is available even in the non-profiled runtime. This breaks down the heap by closure type. Of course, you don't have visibility into the source of the allocation, but this is often still quite useful. — Ben On May 23, 2020 8:58:30 AM EDT, Richard Eisenberg wrote: >Hi devs, > >Is there a way to count the number of times a particular constructor is >allocated? I might want to know, say, the total number of cons cells >allocated over the course of a program, or (in my actual case) the >total number of FunTy cells allocated over the course of a program. > >I can try to do this with an SCC. But it's very clunky: >* The constructor may be used in a number of places; each such place >would need the SCC. >* SCCs can interfere with optimizations. In my use case, this would >negate the usefulness of the exercise entirely, as I think some of the >structures I wish to observe should never come into being, due to >optimizations (e.g. case-of-known-constructor after inlining). > >Surely, the RTS must be able to count the number of times a constructor >is used. But is there any way to access such a feature? If others agree >that this would sometimes be helpful, perhaps we can build the feature. >Now is not the first time I've wanted this. > >Thanks! >Richard >___ >ghc-devs mailing list >ghc-devs@haskell.org >http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs -- Sent from my Android device with K-9 Mail. Please excuse my brevity.___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: profiling a constructor?
Surely, the RTS must be able to count the number of times a constructor is used. It can't. For one there are different kinds of "uses" for constructors. * Allocations - They might be dead by the time we gc the nursery, so the RTS never get's to see them. * Accessing the Constructor? That's even harder to track. * The constructor being present during GC? One can do this using heap profiling (as ben described). There are also top level constructors which currently don't generate code at all (just static data). So currently there is no such feature. For allocations in particular we could implement one on top of the ticky profiler. It operates on the STG => Cmm boundry so doesn't affect core optimizations. There we could for every runtime constructor allocation emit code which will bump a counter for that specific constructor. I imagine this wouldn't that hard either. But it does require modification of the compiler. Cheers, Andreas Richard Eisenberg schrieb am 23.05.2020 um 14:58: Hi devs, Is there a way to count the number of times a particular constructor is allocated? I might want to know, say, the total number of cons cells allocated over the course of a program, or (in my actual case) the total number of FunTy cells allocated over the course of a program. I can try to do this with an SCC. But it's very clunky: * The constructor may be used in a number of places; each such place would need the SCC. * SCCs can interfere with optimizations. In my use case, this would negate the usefulness of the exercise entirely, as I think some of the structures I wish to observe should never come into being, due to optimizations (e.g. case-of-known-constructor after inlining). Surely, the RTS must be able to count the number of times a constructor is used. But is there any way to access such a feature? If others agree that this would sometimes be helpful, perhaps we can build the feature. Now is not the first time I've wanted this. Thanks! Richard ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: profiling a constructor?
On May 23, 2020 9:32:32 AM EDT, Andreas Klebinger wrote: >> Surely, the RTS must be able to count the number of times a >constructor is used. >It can't. For one there are different kinds of "uses" for constructors. This is a fair point. I had assumed that Richard was debugging a residency issue but i suppose that was an ungrounded assumption on my part. >* Allocations - They might be dead by the time we gc the nursery, so >the >RTS never get's to see them. Indeed I thought that the ticky-ticky profiler could do precisely this but it seems I was mistaken; it can only report thunks. Indeed it doesn't seem like it would be hard to introduce this. >* Accessing the Constructor? That's even harder to track. Indeed, this is not possible as written. However, the LDV profiler allows a slightly weaker version on this (reporting whether the closure as been read at all). Allowing full tracking of closure reads this would incur a significant overhead. ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
RE: profiling a constructor?
This would be a sensible thing to be able to count with -ticky. Not hard to add either. (Richard is asking for number of allocations, not what is live at any moment.) Simon | -Original Message- | From: ghc-devs On Behalf Of Richard | Eisenberg | Sent: 23 May 2020 13:59 | To: GHC developers | Subject: profiling a constructor? | | Hi devs, | | Is there a way to count the number of times a particular constructor is | allocated? I might want to know, say, the total number of cons cells | allocated over the course of a program, or (in my actual case) the total | number of FunTy cells allocated over the course of a program. | | I can try to do this with an SCC. But it's very clunky: | * The constructor may be used in a number of places; each such place | would need the SCC. | * SCCs can interfere with optimizations. In my use case, this would | negate the usefulness of the exercise entirely, as I think some of the | structures I wish to observe should never come into being, due to | optimizations (e.g. case-of-known-constructor after inlining). | | Surely, the RTS must be able to count the number of times a constructor | is used. But is there any way to access such a feature? If others agree | that this would sometimes be helpful, perhaps we can build the feature. | Now is not the first time I've wanted this. | | Thanks! | Richard | ___ | ghc-devs mailing list | ghc-devs@haskell.org | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: profiling a constructor?
Thanks for all the responses. Yes, I'm looking for the number of allocations of a constructor; this is an allocations issue, not a retention/liveness issue. I'm not concerned about accesses (not even sure what that would mean). Adding this to -ticky would be very helpful -- and not just for me. For example, it would help us to know more precisely why !2249 (implementing BoxedRep) is mired in performance trouble: we could see how many more TyConApp nodes are created. If this would be easy, is there someone who can guide me how to implement it? I am almost as clueless as someone who has never before gazed on GHC's code in these areas. (Or, I would be grateful for someone else to implement it, too.) Thanks, Richard > On May 23, 2020, at 10:11 PM, Simon Peyton Jones > wrote: > > This would be a sensible thing to be able to count with -ticky. Not hard to > add either. > > (Richard is asking for number of allocations, not what is live at any moment.) > > Simon > > | -Original Message- > | From: ghc-devs On Behalf Of Richard > | Eisenberg > | Sent: 23 May 2020 13:59 > | To: GHC developers > | Subject: profiling a constructor? > | > | Hi devs, > | > | Is there a way to count the number of times a particular constructor is > | allocated? I might want to know, say, the total number of cons cells > | allocated over the course of a program, or (in my actual case) the total > | number of FunTy cells allocated over the course of a program. > | > | I can try to do this with an SCC. But it's very clunky: > | * The constructor may be used in a number of places; each such place > | would need the SCC. > | * SCCs can interfere with optimizations. In my use case, this would > | negate the usefulness of the exercise entirely, as I think some of the > | structures I wish to observe should never come into being, due to > | optimizations (e.g. case-of-known-constructor after inlining). > | > | Surely, the RTS must be able to count the number of times a constructor > | is used. But is there any way to access such a feature? If others agree > | that this would sometimes be helpful, perhaps we can build the feature. > | Now is not the first time I've wanted this. > | > | Thanks! > | Richard > | ___ > | ghc-devs mailing list > | ghc-devs@haskell.org > | http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: profiling a constructor?
Richard Eisenberg writes: > Thanks for all the responses. Yes, I'm looking for the number of > allocations of a constructor; this is an allocations issue, not a > retention/liveness issue. I'm not concerned about accesses (not even > sure what that would mean). Adding this to -ticky would be very > helpful -- and not just for me. For example, it would help us to know > more precisely why !2249 (implementing BoxedRep) is mired in > performance trouble: we could see how many more TyConApp nodes are > created. > > If this would be easy, is there someone who can guide me how to > implement it? I am almost as clueless as someone who has never before > gazed on GHC's code in these areas. (Or, I would be grateful for > someone else to implement it, too.) > Actually, looking more closely at the implementation of ticky, it looks like we *do* track DataCon allocations (the giveaway was the mention of withNewTickyCounterCon in GHC.StgToCmm.Bind.cgRhs). For instance, let con = Just x in ... would produce a ticky counter named "con". The problem is that there is no convenient way to get from the Id name (e.g. `con`) back to the datacon that it allocates (e.g. `Just`). You could of course try to parse this out of -ddump-stg output but... yuck. Perhaps the ticker name should preserve this information. I've pushed a completely un-build-tested attempt at this as !3340. Like most of ticky's output, it won't be the easiest thing to mechanically parse but it should be possible. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
